1. There is no guarantee one of these 'uncontroversial' hard forks won't become controversial, especially as the community gets larger and there are more voices. What was once a consensus position can become contentious due to new information or ideas. This makes the market (people involved in the Bitcoin space) uneasy.

2. Hard forks centralize the community, because in practice, they require the community to coalesce around a small ad-hoc governing structure for the hard fork to have consensus. This makes frequent hard forks antithetical to the decentralized quality of Bitcoin.