This is a valid point and, I don't know, maybe we should not wait for a "divine intervention" to fix this (although maybe a differentiation would be needed between trusting and empowering users).
Because while some fight each other and may have both good intentions (yes, I know that in this context this statement can explode), some real trolls simply get stronger. I won't tell names since I don't have solid proof, so it's just a hunch.
Maybe a new topic in Meta is needed to get ideas how to use better the current trust system. One idea that came into my mind was to select carefully a very short list of users we want to "empower". Or maybe it should be as wide as possible?!