Bitcoin isn´t big/stong enough to really disturb government and big banks, probably a minor incovenience by now, we see some talks about regulation here and there, but nothing concrete.
But as it gets more popular and a direct impact in their operations start to appear, it´s reasonable to think action would be taken, even making it illegal.
Sure they won´t be able to seize our bitcoins, or stop transactions, but this won´t matter, the support and general belief will suffer major damage, price would crash to hardly recover ever again.
US took down major and global poker sites overnight due to suspicion of illegal activities around it, why should be different with bitcoin ?
For now why should we be afraid after some countries has legalize bitcoin maybe where i live now bitcoin is still not legalized but i think there is a possibility it will happen when many countries has legalize bitcoin.