The government tends to abuse their power, and what we see is only a fraction of what is going on. Spying on everyone isn't going to help their "terrorist fight" or whatever.
I am yet to see a single evidence to support the argument that this mass surveillance has reduced the terrorist activity in any way. If it was so, the Boston bombings, in which a number of innocent people were killed would have never happened. The perpetrators were under the FBI watch-list, and only a few days before the bombings, the FBI had received a warning from Russia about them. Seems like the CIA / FBI was spying on normal people, and had no time to do surveillance on terrorist suspects.