He gave you negative feedback simply for distrusting him ? I don't know the history between you 2 but kind of agree that it's messed up.

Also what impact does the trust/distrust hold other  than being able to see more clearly the people who get tagged by them ? Is there something else that I'm missing ?