I think that the main problem of bitcoins (and that must be ranked as the number one of downsides) it is that he is not physical money. This make bitcoin usable only in internet and the ways that the money can be used in internet are limited. Then not to many people use internet and are even fewer those who make purchases there.
Actually there are issuers of Bitcoin physical coins, but these physical Bitcoin coins are not popular because you have to trust the issuer and it goes against the trustless idea of Bitcoin. Some alternative could be hardware wallet with wireless comunication to pay in store in future, but for today you can use Bitcoin debit card - although this way you support credit card company fees which goes against the low fee idea of Bitcoin