Nice concept. I don't like storing the wallet on the server tho. Also in some ways wouldn't this reduce anonymity? Transactions correlate wallets with a certain IP address every time...
It's called "server" only because it implements the Electrum server protocol. In practice you'd run it on your own hardware, generally in the same place where your full node runs so the anonymity would not be harmed in this respect.
OK, but then you're back to leaking your IP address again.