Correct. Let's say the block limit becomes 20MB and some spammer was capable of filling up three of them in a row, delaying confirmations for 1/2 an hour. In which case, the same spammer could right now, fill up 60 blocks at 1MB, and delay confirmations for 10 hours!  Worse, once the 1MB blocks are normally 75% full, any spammer with that capability could probably cripple tx confirmations for a whole week!