The rationale is that something which isn't connected to the network is less risky (malware, virus, keyloggers...). But note that even an usb device temporarily connected to a cold wallet computer might be considered as a risk.

A paper wallet has a big advantage : you can't connect it to the network  
But paper wallets come with their own challenges: you must store them in a secure place (thieves, water, fire, ...).

I guess this is why so many people are excited by hardware wallets which are a good compromise between security & convenience (but you still have to store the "seed" in a secure place).