Decentralization is not an important goal in and of itself.  The end goal is censorship resistance.
User adoption contributes to censorship resistance. Therefore to the extent a block size limit restricts user adoption, it also diminishes censorship resistance.