Sure, they can do that, but the damage will already be done.
* Currently only 10 major pools to choose from.*
The Powers that could collude 4, could collude all 10.
Two colluding partners is difficult to maintain (since either partner can gain an advantage by first agreeing to collude and then operating outside the collusion).
More colluding partners is even more difficult to maintain. There is more opportunity for a partner to operate outside the collusion, and more opportunity for the secret to leak.
Even if you could get 10 pools to collude without cheating on each other, there's nothing stopping me or anyone else from starting up another pool.
The troll discounts the fact that miners/pools/ASIC-producers have a direct long term incentive for the success of the network.
Plus if they were truly colluded, the troll discounts the fact the Bitcoins is composed of people, who can be in consensus to fork the network away from anyone who tries to co-opt the network.