Same I keep hearing newbies should not be doing this, yet on the other hand people say its a secure way of storing your bitcoin..

I cant see what problem can occur. Simply send Bitcoin to the generated public key and when you want to access the funds simply sweep the private key on a site that has this function.

Can anyone provide any caution? Or possible failure points?