If someone had enough hashing power to build off their own chain (aka, 51% of the network; or using selfish-mining principles, as little as 30% of the network), they could predict the nTime required on the next block that would result in the following block being an anomaly.  Using this system, and by only building on your own chain, you could cause EVERY block to be an anomaly.  A would-be attacker (say, multi-pool), could mine the network, orphaning everyone else's blocks, and get a payout on every block at the same time.

Furthermore, there is absolutely no reason to mine worthless blocks (non-anomaly) - if you wanted to mine CGA, it makes the most sense to mine litecoin (for example) until the next block is an anomaly, and then switch to CGA, switching back as soon as a block is found by the network and the next block is not an anomaly.  The diff is low enough that solo-mining would likely find this fairly effective and profitable.

If pools, or someone solo-mining felt like it, they could also pick nTimes selectively and cause the following block to always be an anomaly (wouldn't necessarily benefit them, but would significantly increase the number of anomalies that occur).