Decentralization is not an important goal in and of itself. The end goal is censorship resistance.
User adoption contributes to censorship resistance. Therefore to the extent a block size limit restricts user adoption, it also diminishes censorship resistance.
How does user adoption, in isolation, increase censorship resistance?