By the time the blocks are 20mb each we should have no worries on space cost, if it is raised it won't magically make every block 20mb just allow them to go that high.
Exactly. The existing 1MB limit is not keeping the blocks at an average of 30% to 40% full today, just as it was not keeping them 10% full three years ago. If the block size limit became 20MB tomorrow then it might still be four years before the first 20MB block occurs, and five years before they are common.
One thing I wonder/worry about is miners deliberately mining small blocks when the pile of unconfirmed transactions is waiting.
This worries me too. It is a serious risk. Once the blocks are an average of 75% full the network goes into a red-zone where a bunch of deliberately small or empty blocks can pile up the unconfirmed tx causing delays of hours for confirmations. It is a very cheap attack. Just sent some coins to an exchange to sell? Come back tomorrow if this attack is in progress. Yet some people are more worried about a few 20MB spam blocks being mined, which the network would hardly notice.