I believe it should be high priority for the Electrum system to show unconfirmed payments as rapidly as possible, so as to make for a Newbie friendly user experience.
I agree to an extent. However I think the problem is mitigated for "newbie users" because they will likely use "auto connect" and I'm assuming lagging servers will not be autoconnected to.
Of course the electrum server software could probably be made to perform better, but I'm guessing it'd be quite a bit of work.
Until then and although it's a bit sad and maybe even mildly concerning (centralization), maybe your little atom (not to discredit it, but I assume it's the bottleneck in your system when electrum is digesting transactions) will have to leave the work to faster machines regarding servicing the masses?
A couple of years ago, I played around with Electrum and rejected it because of the "server lagging" issues and the unreliability of connections to the servers. It seemed dodgy. These problems are not new and not the fault of my particular server hardware. At the time I was not running a server and was using the auto selection feature.
I chose the little Atom deliberately, so that it would be a canary in the coal mine. My little machine is not servicing the masses. It's just servicing me. If tracking the block chain can not be done efficiently, then the future of Electrum is at risk. I would say the future of bitcoin as well as a decentralized system, except that the little Atom manages to keep up with the present load using only a few percent of CPU time to run bitcoind.
Electrum-server used 5832 seconds to process one block! This is a bug. Not a slow processor.