block size does not harm decentralization, how block size is increased can do that. for example if you increase it suddenly to a much bigger number now it will end up centralizing bitcoin but if it is increased slowly with the advancement of hardware and internet speed then it can't do much harm.
There's still a larger problem here that most people don't discuss. Even if hardware advancement allows us to significantly increase block sizes over time, that doesn't address whether doing so is compatible with Bitcoin's hard cap on supply.
Without inflation, the system needs fee revenue to continue incentivizing miners. The block size limit is the only means we have to enforce scarcity of block space, which guarantees fee revenue. Otherwise, Bitcoin's Byzantine fault tolerance may be threatened as block rewards decline in value. You can't have a network worth many billions or trillions of USD where miners have no incentive to secure the network.