Nice concept. I don't like storing the wallet on the server tho. Also in some ways wouldn't this reduce anonymity? Transactions correlate wallets with a certain IP address every time...
It's called "server" only because it implements the Electrum server protocol. In practice you'd run it on your own hardware, generally in the same place where your full node runs so the anonymity would not be harmed in this respect.
OK, but then you're back to leaking your IP address again.
Also from the README:
Electrum wallet must be configured to connect to the server. SSL must be disabled 
Pull requests welcome ¯\_(ツ)_/¯
All you have to do is read
this page and figure out how to make the server socket be SSL. Since you're generally only connecting to localhost when using Electrum Personal Server, there's no privacy or security difference between SSL and non-SSL.
I agree there is no point in using SSL to connect with localhost.