Can you not make a new wallet?

You can keep more than once wallet with electrum and a new one will greatly speed up your access to some funds if not all for when you want to sign or load up the wallet.

I'm guessing the slowness is because your wallet is encrypted and the node will slow you don't because of the time it'll take to confirm your request there and I wouldn't suggest not keeping an encrypted file so a new wallet is probabky the best way to go...