It's more of a being sure everything is fine outside the offline wall thing. Just to be sure I'm still the owner of those funds

So the only way to be in peace that everything is fine with my cold wallet is to check with my master public key?
Yes, that is the best. But, while creating the electrum cold wallet, did you created watch-only wallet along? You can use it to track your balance as well. Or, you can check it on blockchain explorer. But, checking it using watch-only wallet is recommended.
Use the link below to be able to create the watch-only wallet:
https://electrum.readthedocs.io/en/latest/coldstorage.html