Does that bother anyone ?
Yes and no.
Bitcoin still has a big peer-to-peer network of full nodes that enforce the same rules - if miners would produce an invalid block, like stealing someone's coins or printing coins out of thin air, then this network would reject this block as invalid. There are attacks that produce valid blocks - double spending and partial/full DOS. If such attack will be detected, the full nodes will have the power to fork to a new algo to essentially fire all legacy miners forever. Bitcoin is relatively safe for as long as the network of full nodes is decentralized and has low entry barriers (low blocksize).
However, I don't like the fact that miners create shitcoins with the same PoW and hop between them to profit from difficulty adjustments, as well as attacking Bitcoin to boost the value of those shitcoins to increase their mining rewards. If mining was decentralized, this wouldn't be a thing.