the ones who control most bitcoins has more to lose.
Miners aren't necessarily the ones who control most bitcoins! The early adopters control a large amount, the future miners have less than one-third of bitcoins left to mine the rest of this century.
"Controlling most of bitcoins" and "controlling Bitcoin" are two totally different things. Satoshi, for instance, controls around a million BTC, but he doesn't control Bitcoin's day-to-day functioning any more than the guy with .01 BTC, the miners do tho. They stop -- no more Bitcoin. That's why the halvening is going to be so interesting: either the price of BTC has to spontaneously double, or roughly half of the miners have to close up shop.