There's a label backup option under "Wallet->Labels->Export" menu.
That will produce a .json" file with default name of "electrum.labels.json" which you can rename.

Then you can import it to the other instances of that wallet using "Wallet->Labels->Import" menu.