The non-aggression principle can be applied or ignored by both atheists and religious people. There is no monopoly on rationality and life is simply to complex to claim that you are rational on all fronts. Remember one of the least rational and most destructive systems ever was sold as "rational" and run by atheists, unless I've just misread this whole thing about the soviets and they were all closet Catholics.
Currently the truest and most consistent libertarian with any political power is a Christian named Ron Paul.
The problem that I see with most secularists is that they don't see that secularism is just as much a religion as any other.
And I find individual fundamentalist secularists to be at least as annoying as fundamentalist christians or muslims or jews, maybe a bit more. In aggregate though, they are much worse because it is apparently socially acceptable to loudly display secularism as a status marker in bars.