You are right! And you need to trust Thomas V! He is only a human and can change his mind...
If you are this much paranoid then you can compile Electrum on your own since it's open-source. I'm quite sure that other developers of Electrum sign the executable file with their own private key, but I can't find any example right now. Oh wait, Microsoft has acquired GitHub so by that line of thought, you have to trust them as well.