I don't think the kooks are a majority, they're just very vocal. The trolls are as big of a problem as the kooks are. Normal people are definitely interested in Bitcoin, we just have to shut the kooks and trolls up so that the normal folks will tend to stick around.
I see far more people calling everyone who doesn't agree 100% with either their politics or their rose-colored view of Bitcoin trolls than actual trolling. And even for actual trolling, if people responded with intelligent, logical arguments instead of kooky crazy-talk, it would be a very effective way to persuade casual lurkers than trying to ban all comments that aren't reflexive agreement with the majority view.