Currently someone could jump on the network with a big hashrate and gobble up 20 blocks until the difficulty goes up and then leave the other miners to struggle with the higher difficulty. Wash rinse repeat. right?

It could be a wise decision. Any drawbacks though?