More importantly the network does not cooperate with any attempted double spends. Right now, the first attempt at spending a specific coin will be the only one that gets relayed, any subsequent attempt at spending it is dropped. There is talk of actually spreading a warning when a double spend is attempted, so that not only would you not be able to double spend the money, but also the merchant would be quickly warned that you attempted to do this.
https://en.bitcoin.it/wiki/Double-spending has a lot of details on different threat models and how to avoid them. General rule of thumb is zero conf-doesn't take a long time to be safe, as long as the value of what's being sold is low relative to the value of a block.