In most brick and mortar stores, I don't think there's much of a problem with double spending. In order to pull it off, you'd have to immediately stop after buying your physical goods and try to whip up a new transaction--probably sending coins back to yourself--and somehow broadcast it louder to the network than the merchant's POS device. Chances are the timing will not work out in your favor. Zero confirmation transactions should be fine in most cases.
If I pay a merchant with no fee attached, and then step outside the store and double spend the same coins with a large fee attached, then surely the second transaction would get into the chain first. Am I wrong?
Surely, no. Possible, yes.
However, there are many places in the world where scamming a local store will quickly make all merchants in the area refuse to deal with you except possibly on a cash-in-hand basis. And some won't even deal with you then as a matter of principle. I know it sounds strange to those of you who hide in the crowds of cities, but us rural folk don't have the luxury of being dishonest and getting away with it.