Thank you Dave. I may sound dumb but honestly didn't know you can connect SPV wallets ,other than Electrum, to an Electrum server. Thought there would be some compatibility issues!
But, others do not such as Mycelium and Green.
for Mycelium,
giszmo said that you have to change the server's address in the source code:
You could already for personal use replace our url with yours in the source code.
We currently use a slightly patched electrumX server. For almost all users the patch is irrelevant but for those with gigantic wallets or very slow internet the patch matters.
source:
https://amp.reddit.com/r/mycelium/comments/b3n8ww/any_plans_to_add_the_ability_to_connect_to_your/I was also able to connect a my Green wallet to a random Electrum server and it synced without any problem. thus, I believe you should be able to connect it to your own server too.