In my opinion, the Electrum wallet is more vulnerable than the hardware wallet;
Because you are using the same device, it must be online to make and broadcast transactions.
Because you have set the hardware wallet as one of the cosigners on the multi-sig wallet, if you don't connect it to Electrum, then the wallet can't be used.
So let's say while the hardware wallet is connected in Electrum, then Electrum somehow gets compromised, the hacker then sends a transaction; they still couldn't spend any coins unless they have physical access to the hardware wallet (ledger) to sign and broadcast the transaction, right?