You're right that I don't often change my Trust list.
If you are using the system as the way you should (you do of course), you don't need to change the trust list every now and then. I have included users in my trust list because they provide feedback which I think worthy to look at when I trade. I'm also don't change the trust list every now and then. There was an issue earlier which forced me to edit my trust list. Though I'm thinking of going with Default Trust instead of having custom trust list as it creates some issue sometimes. I don’t want them at all.