Here's an experiment to try with your friends and family to see if PoS is viable. Play the game Monopoly by Parker Brothers with a few small rule changes. You would play with one die per turn because shaking two dice takes more energy (we can't have that) than one. We remove the rule about shaking doubles for extra turns because too much variance is just silly in a PoS game. Then, compare your real life net worth with them and you each get to take a proportional number of turns by ranking order. For instance, if you as a childless adult have five as much wealth as your poorest married competitor, you would get five turns at the start. After the first round you then get another roll each round for every property and house that you own. After playing one game, see who would play the game with you again. Most likely the winner of the game would be the same every time.
PoW does have a real threat that if someone gets too much control of the network they can reject transactions until that monopoly is broken, but it can be broken and then things go back to normal. With PoS, once a monopoly takes hold it would be nearly impossible to change the balance of power. As a monopolist, you could dictate who does business with whom. You can even choose who gets to buy food for their families and who shall starve to death. That would be fine for the people in the good graces (rhymes with races) of the monopoly holder, but it's more likely that everyone else will simply switch to another currency and not play with you anymore.
There is a reason that Satoshi created variance for block rewards. The element of chance (like playing the game Monopoly with an equal amount of dice) adds enough chaos to make the balance of power unpredictable. Some may not agree, but most people believe that everybody deserves a chance in life to thrive. When the block rewards are sufficiently depleted to remove variance, there should still be a mechanism to add variance to fee rewards. Just like the game Monopoly (one of the most popular in history), people will continue to play a PoW game as long as they have a chance of winning. I wholeheartedly disagree that downtrodden people will succumb to a monopolist or we would all still be speaking Latin. There must always be a fighting chance.