SegWit have no downsides while increasing the blocksize could give us centralization to a certain degree as the blockchain size increases over the years.
any kind of change will always have both downsides and upsides. so it is wrong to say SegWit didn't have any downsides! because it did. for example the fact that it is opt-in makes it an inefficient block size increase. and also the complexity of it has been a problem for services to adopt it.
Segwit's goal wasn't the blocksize increase, so it's wrong to criticize it for it. And being opt-in is the result of softfork, and softforks are great because they are much less disruptive than hardforks, so there's almost no risk of chain splits.
It's also wrong to say that SegWit was complex, all you had to do is update your wallet software and create a new SegWit-compatible wallet. Some services did this only after a few days or a week after the fork, big services like Coinbase just didn't care about their customers, which resulted in huge fees later in 2017.