I think that the main problem of bitcoins (and that must be ranked as the number one of downsides) it is that he is not physical money. This make bitcoin usable only in internet and the ways that the money can be used in internet are limited. Then not to many people use internet and are even fewer those who make purchases there.
Shops can have WIFI connection which allows users to broadcast their transaction using that from their smartphone. Not a high connection speed is required. Alternatively, a service can create a card service and allow users to load their account with Bitcoin and it would act like a debit card, allowing users to swipe it at stores.