Have you tried the "ignore" function?  With some discipline, I am able to remove low-IQ contributors, whilst keeping most of those who have something to contribute - even if I totally disagree with it.