We were analyzing scenario when harvesting is being done on a computer that may belong to an adversary. Let's name person who harvests as Alice and the owner of the server as Bob. If Alice doesn't have a copy of the whole blockchain and a fork has happened then she can't decide which branch of the two is legit. Bob may use this to trick Alice into harvesting on his secret branch trying to do a double-spending.
Your reasoning is perfectly valid with few buts...
1) he would have to convince "quite a few" users, to actually make this successful. If one person is able to do that we're doomed anyway (same in NXT)
2) it does not work like "power leasing", so attacker actually needs to generate block using every key == waste of time and resources, while meanwhile main chain is going forward...