Ignore would be great. Voting system, not so much, as it could be easily abused. Just think about groups of trolls working in tandem, and how they could really screw things up with a public voting system.
Some sort of rep system could be set up... But I agree it could be abused, if not properly managed or weighted. Might be more work than it's worth.