It does take a bit of trust to be among the first to install a new release, but you can also verify for yourself that the download is safe.

First and foremost, you should learn how to verify the download using ThomasV's PGP signature.  This will ensure that you've downloaded the official version signed by the developer.  Even if Electrum's website gets hacked, and the download link is directed to a malicious version, this will mitigate your risks.  Here's a tutorial I wrote: [GUIDE] How to Safely Download and Verify Electrum [Guide].

If you still don't trust the binary releases, you can always take some time to verify the code hasn't been manipulated with nefarious intent.  If you go the Electrum repository, you brows through the changes made in real time.  You can confirm for yourself that only the changes published in the release notes are implemented into the code.

And if you're still doubtful about the binary releases, you can even run electrum right from the source code.  It's really if you're running Linux and have all the python dependencies installed (which you probably do if it's a recently released OS.)

Electrum's github.com repository: https://github.com/spesmilo/electrum
Electrum's Release Notes (change-logs): https://github.com/spesmilo/electrum/blob/master/RELEASE-NOTES
Electrum's previous releases: https://download.electrum.org/