...and also people can use different types of pool software such as p2pool etc which offer different features...
The centralization of pools is a problem until it is solved, apparently p2pool is not the solution, and this sort of hand-waving and saying "it'll be ok, we'll figure this out" strikes me as naive, and fundamentally unwise.
forgive my ignorance, why is p2pool not going to work?
because he is mis understanding the multiple users connected to a mining pool, centralisation (which i and the OP mentioned) vs a single mining farm centralising the hash power.
The traditional pools could all cease to exist today and Bitcoin would be just fine. There would probably be a period (a few days to a week) of slower than average blocks as unprepared hash rate providers set up p2pool and solo mining (becoming actual miners in the process). Any competent hash rate provider should have his mining software already set up to automatically default to p2pool and ultimately to solo mining should the traditional pools fail for whatever reason. Mining software makes this extremely easy.
I mentioned in the OP that P2Pool can not be resistant to the share withholding attack:
...
Some of have argued that there is P2Pool, but how many times do I have to repeat that P2Pool can't be impervious to a share withholding attack[1], thus it can't be a sustainable paradigm.
...
See the linked posts from the [1] footnote for more explanation. In a share withholding attack, the adversary sends shares to the pool so he gets paid, but withholds any share that is a block solution, thus the pool never gets any revenue from the adversary's shares. Thus the adversary steals income from the other members of the pool. The government can entirely destroy P2Pool if ever it becomes widely used.
Share withholding can in theory be used to destroy all pools in Bitcoin now, because Bitcoin did not implement Meni Rosenfeld's oblivious shares fix[2] (which he proposed I think in 2011). Even if Bitcoin or other altcoin implements Meni's oblivious shares fix, P2Pool can not use it, while server based pools can. The reason is because the secret (so no miner knows which shares are block solutions) in the oblivious shares fix can't be hidden because in P2Pool there is no pool server to hide it with.
So franky1 et al, this should serve as an example to you that in software engineering, the details matter a lot and can totally change the conclusion. So please check your hubris and confidence at the door before you enter this discussion, because I have done a lot of research and thinking about the details that matter. I appreciate our discussion because I want to determine if I am incorrect. And I will surely admit it if I am. But I also hope you approach this discussion with a similar open mind, because it is also likely that you are incorrect.
Additionally P2Pool requires that every miner run a full node, which given the size of the block chain (and even savvy forum members complaining about the download time) for Monero and Bitcoin, that pretty much rules out mining by all and certainly rules out instantaneous mining and gratification (which is one of my main goals for an altcoin and please don't tell it is not possible and botnets and all that crap because you don't know the details of what I have up my sleeve). Remember my special talent is not that I am not just a programmer, but also a capable marketer (I had 1 million users of my CoolPage.com software in 2001, when the internet had 1/10 the users it does now). Also most ASICs are not capable of doing the logic of a full node[3].
Additionally P2Pool would make it unlikely to achieve fast block periods[4], because the propagation delay would be quite variable given so many different cases of peer connection qualities.
[2]
http://arxiv.org/pdf/1112.4980.pdf#page=29[3]
https://blog.ethereum.org/2014/06/19/mining/[4]
https://blog.ethereum.org/2014/07/11/toward-a-12-second-block-time/