If you keep Bitcoin 1MB block limit, people will be forced to use other altcoins as well, so you basically have to download all blockchains anyway because not everyone is going to accept Bitcoin, so you save nothing from this point.
In your logic you could download only what you're interested in. Not *all* chains.
Which is actually better than having to download one massive bloatchain.