The chances of having a transaction of 3+ confirmations rejected or double spent isn't something I would worry about for normal Bitcoin usage. By normal I mean receiving $100s or even $1000 worth of BTC. If I were to ever receive millions or billions one day, I would wait for at least 6 confirmations. Not sure if that will even happen though.    

1 confirmation can be problematic and it doesn't have to involve a scam or double spend by the other party. The block could be orphaned. You see 1 confirmation, you release the money or goods, and the block your transaction got mined in gets rejected by the network. Then you are back to no confirmations. You weren't scammed and probably your transaction will be mined just fine in one of the upcoming blocks.