Note that every wallet (which is not a full node client) can gather these information.
The only way to circumvent this is to either (1) run your own electrum server or (2) use a different (full node) wallet, e.g. bitcoin core.
On a side note, if you don't change your wallet/address & only run electrum server, another electrum server you're connected could use logged information to track you.
It also apply if you simply send all your Bitcoin to different address/wallet.