Simply put - BTC can't go further adoption-wise without these big fellas, smaller institutions, and some govs making something with it.
It's natural and they wouldn't be able to harm it in a big way or for a long time, as long as there is no team-up with the exact cause of doing something bad towards BTC or crypto, in my opinion.
This is the natural flow of adoption, we cannot please everyone to buy bitcoin store it, and use it in the future so here comes the huge ones playing around the crypto. As they see it is being accepted in most countries, they will try to own as much as they want driving price and adoption. But essentially, OP is quite right, if most of bitcoins are owned by these big ballers then sure bitcoin is not a game changer anymore which will then we can go back to traditional one leaving them a useless digital currency to play with themselves lol