The government.
I don't believe that governments are doing anything at the moment to stop people using bitcoins. Even the most strict governments as China and Russia still haven't done anything major that would cripple this technology.
Yes, everybody has issued warnings, etc, but this is nothing compared to what they can really do if they want to, the governments.
I don't think the government has any problems with bitcoin. They don't see it as a threat to what they are doing, that being running their our countries.
The problem lies with the people that use it and what they are using it for. If it is against their laws then of course they will come down on those individuals just not by the means on which they came to possess such things.