You don't have to care about them, but if the CIA/FBI/whatever sees that bitcoin can be a thread to the mighty US dollar, you bet your ass they will do anything they can to stop it. Look at the US sticking their nose anywhere and everywhere the past 100 years. And with the amount of money they have, they can definitely perform a 51% attack by building a shitload of ASICs.
You know, now that I think about it, it would be trivial for the government to indict an ASIC company right before they start shipping. Through the discovery and asset seizure processes, they'd be able to snatch all the ASICs and pre-order lists and do what they please. This would be a bit more nefarious from the government's angle, but I assess it as a higher possibility than collaboration between an ASIC co. and the government from Day 1.
On the other hand, collaboration from Day 1 would be a higher likelihood if the government already had an upper hand/bargaining position on an owner of the company (e.g. the person is a felon, on probation or parole, former military, or something along those lines).