Bitcoin isn´t big/stong enough to really disturb government and big banks, probably a minor incovenience by now, we see some talks about regulation here and there, but nothing concrete.
But as it gets more popular and a direct impact in their operations start to appear, it´s reasonable to think action would be taken, even making it illegal.
Sure they won´t be able to seize our bitcoins, or stop transactions, but this won´t matter, the support and general belief will suffer major damage, price would crash to hardly recover ever again.
US took down major and global poker sites overnight due to suspicion of illegal activities around it, why should be different with bitcoin ?
Government wouldn't take a drastic action has declaring bitcoin illegal unless it threatens their mainstream income, what they can do is give bad press to discourage their citizens from using bitcoin.