I think that many people, versed in pre-cypherpunk organizational modalities constantly seek to attribute control to some authority in any system. When they look at Bitcoin they see mining and say "ah, here is the parties that are in charge" (or, alternatively and less  often, some software developers).  They're wrong about that: no one is simply "in charge", and that is a big part of the point and value. It also makes some people uncomfortable, because what they want is the old way of doing things with singular authorities which can be influenced in traditional ways... rather than the personal autonomy of cryptographic proof that backs Bitcoin.

The reality is miners share power with many other groups and there exists a complex interwoven power dynamic between nodes, users, merchants, processors/banks/exchanges, developers, and miners. When developers discuss centralization of mining concerns among each other in private there is an understanding of these complex nuances already .