another way as a comment above, I never try it before, but electrum had also a plugin function for the synchronization label. So when restoring the wallet from seed, the label will show up without importing the wallet file.
I don't think it's a good idea to keep labels, addresses and transaction on some remote server even if it's encrypted, but it may be better if you run your own server.
There's a label backup option under "Wallet->Labels->Export" menu.
That will produce a .json" file with default name of "electrum.labels.json" which you can rename.
Then you can import it to the other instances of that wallet using "Wallet->Labels->Import" menu.
I exported my labels with that option and I also made a regular backup before, but I wonder if there is any difference and what else is contained in regular backup file (except labels)?
Maybe it is transactions, saved contacts and something else, but I couldn't find any information on electrum website.