The biggest risk to bitcoin comes from the US government, which may criminalize and ban it. In fact I am pretty sure they will do that once it gets into their radar.
I'm 99% certain that your prediction will happen. The Government hates competition, they love forcing you to use their counterfeit money.
However - and I am not joking when I say this - I CAN'T WAIT till it's illegal. Because once it is, the mainstream will find out about it.. and THAT's when more people will really start using it, and after a brief panic-induced selling from wimps, the price will probably skyrocket. Even though it wasn't money, that's essentially what happened with Napster, Bit Torrent, the whole p2p movement in general. Once they make it illegal, that's when people will start using it.
Additionally, it's irrelevant what the USG does. There are 200 countries in the world, the USA is one of them... that works out to 1/200th of the world. In terms of population, roughly 300M out of roughly 6.7B. That's just 4% of the people in the world. What I'm saying is - when they make it illegal, nobody cares. When other countries make it illegal, still nobody will care. People will just move to VPNs and alternative exchangers. Life goes on. Don't forget that the Government is there to serve US, not the other way around. You are only a slave if you accept to be one. You can be a free man if you decide to be one.