In your 10,000 example above, you'd have to process at least 10,000 challenges per voting session, and you still only get 1 vote in that session per "node".
Each voting session will generally last between 30-60 seconds depending on whats happening network wide and if there are any conflicts.
If there is one challenge per vote, you have this 10,000 ports attack mitigated to some degree; but I'm confused why you even need the trust stuff on top of this, I don't think it adds anything?
edit: but lets move this discussion to an emunie thread so as not to derail this one further