If you are going to argue that who people should trust should be determined in a decentralized way then I would say scammers would manipulate this and exclude who is trustworthy and include fellow scammers. There are a large number of scammers who have excluded me from their trust network while they have added their alts and fellow scammers to their trust network. The fact that scammers outweigh honest people (especially when it comes to the number of accounts) means that any voting system will result in scammers appearing as trustworthy and honest people appearing as scammers.