He can't just sit and produce blocks forever. In order to be able to produce blocks he must keep balances under his control. First he must exclude transactions emptying his cheaply acquired priv keys, then he will probably want to transfer balances from that keys to his own keys, because if he doesn't do that those gullible large stakeholders who sold him their empty keys will be able transfer funds from those keys again to their new addresses. In any case, attacker's fork will look completely different.
In that sense you are correct, yes. But the attacker would be wise to just censor the transactions sending his funds away and just keep on trucking.
He will need to censor also all transactions that depend on transactions sending his funds away, and that will soon be a lot of transactions.