The public key is a subset of the private key.
The bitcoin address is the hash of of the public key.

This means that your public key and bitcoin address can be derived from the private key.

The electrum seed generates a series of private keys.
This means all the private keys of a wallet can be generated from 1 seed.

The electrum master public key is derived from the seed.
All the public keys can be derived from it, and in turn, all your bitcoin addresses.

Conclusion:
- Just backup your 12 word seed. (13 words for electrum 2.0 HD wallets).
- Do not export private keys of an electrum wallet unless you really know what you are doing. There are tools that derive all your other private keys based on a single electrum private key.
- Use a watch-only wallet based on the master public key for wallets on PCs you use a lot.