A lack of knowledge about even the existence of Bitcoin keeps a majority of consumers away. For the rest of us - there are some who know but don't bother simply because it's more effort to buy Bitcoins and then spend them than just use your credit card to buy whatever you want to buy.
Well for me they just don't know what bitcoin is and they don't believe that bitcoin is really a true money at all. But supposedly if they are going to know what bitcoin really is for sure they are going to start using bitcoin for sure. But we can't force them to do it so, what we need to do is to educate them what bitcoin really is.