You do realise that a block as big as 64 MB will take a lot longer than 10 minutes to verify on anything but the most costly hardware?
Actually this can happen at 2 MB block size limit due to the problem of quadratic scaling. Segwit aims to improve on this problem by scaling it down to make it linear. Gavin does acknowledge this as his own proposal (BIP 109) contains a workaround that prevents this from happening (sigops limitation).
Was this limit put in to prevent people from spamming transactions until there was enough network hashing power to cope with an increase if I'm understanding this correctly?
No. The problem is not directly related to the hashing power. You can look at three factors: 1) Storage; 2) Bandwidth; 3) Process power (for validation).
Thanks for clearing it up I won't pretend to be an expert on this, it's not my area at all.