Wouldn't it be dumb of you to cycle merits along the forum, while the administrator has chosen you as a merit source? Unless you don't care about your reputation or the forum at all, there's no reason to abuse your merit source position this way.
Usually, Merit sources care about the forum and their reputation. I've seen theymos undo some sMerit sent by a source, but it's rare.As theymos says: "it's not a big deal":
Aside from that, if people complain about whether things deserve merit at all, then that's something to perhaps think about, but if you conclude that they're wrong, then that's that. You don't need to stress about it or defend yourself constantly. It's conceivable that someday you and I will end up disagreeing too much about this stuff and I'll remove your source status, but it's really not a big deal.
(There are also accusations of sexual impropriety, but those accusations aren't always false. 
)
I'm still waiting for my copy of the video.)