It matters because miners do not have an economic interest in forcing end users off the network. End users provide fees and demand for the currency which is how miners make money.
But if BU will switch to the longest chain eventually anyway, then what exactly is the point of your limit?
You can't blindly accept a 2 Mb block just because it's below your limit, as it might be orphaned because no other miner agreed on it.
So you will still have to wait like 6 blocks.