Yeah, I imagine it's pretty annoying especially for those in the running of winning something. I bet it could be quite easily done to measure the effect of the merit awarding, since there is definitely a reoccurrence of certain users since they have been giving merit, which didn't seem to be in the running at the very start. Although, that could just be exposure, and natural variation, so it probably wouldn't prove much at all. Although, I do believe the OP has the ability to not include anyone's votes at their discretion.
It really seems strange to see candidates giving merit to those who vote for them. But as it is impossible to block attribution of merits in a specific topic, only the common sense of each user remains. Perhaps you could do it at the end of the poll, rather than during the poll, to try to avoid becoming an influencer.
The only alternative would be for the votes to be secret, with the OP having to accept the DM votes. But that would be crazy for him to manage all the votes and it would take up a lot of time.
Unfortunately, I don't think there's much the OP can do. Just trust the common sense and honesty of other users.
Either way, that's not what will tarnish this initiative, which has as its main objective to animate the community and involve us all.
