If you are patient enough to wait for at least one confirmation then you are no longer vulnerable to race attacks or Finney attacks. Now your only concern is 51% attacks.
Actually, 51% attack isn't your only concern. I'd say that an attacker who wants to deliberately reverse your transaction is a small concern (of course depending on the amount of money you move). The common concern with 1 confirmation is occasional reorg. It does happen once in a while, inevitable in a decentralized system. And once your transaction suddenly turns to unconfirmed from 1 confirmation, things go nervous.