If we start raising the block size it will be like that in a way: tomorrow 2MB, in one week 4MB, you understand the trick.
Simple and straight to the point. The end result would be simply that users on slow networks will be unable to catch up.
At least there will be something to catch up to and a reason to upgrade one's infrastructure if one needs to run a full node or wants to watch Netflix.