I am also WAY more trusted than he is

Your trust score has nothing to do with your ability to use the trust system appropriately.
Think about it for a second: how does people leaving you positive trust reflect your ability to exhibit a good sense of judgment about others?