I've had issues in the past with Electrum when trying to switch from one hardware wallet to another hardware wallet without disconnecting the device... where the device won't be "detected" after trying to switch wallets until I shut everything down and then start again.
Not just with multiple hardware wallets, issues with Electrum occur if you try to switch between different wallets (Ledger bitcoin accounts). If I use my Ledger together with wallet1 on Electrum, I can't simply click on File > Open in Electrum and open up Wallet2 for example. There is always an error about the device not being recognized. I have to first close Electrum, restart it, and then open Wallet2 for it to work. The combo Trezor-Electrum could cause the same type of connection problems.