Hardware wallets have never really been very good at dealing with large numbers of UTXOs as they pretty much all run using relatively low-end hardware with the associated memory/CPU limitations.
What might be "fine" for you average PC/desktop, can really cause issues with hardware wallets as they struggle to process/verify all the data from 10s/100s of transactions etc.
I get that it takes some processing power to sign the transaction, but how hard could it be to design it in such a way that it doesn't matter? Especially if the same address received many transactions, Electrum handles all inputs, and only sends the ones that need signing to the hardware wallet. So even if you have 10,000 inputs, I don't get why signing a transaction with 10 inputs would take longer than it takes when you don't have those 10,000 inputs. To compare: If I would copy the same unsigned transaction from a watch-only wallet to sign it offline with Electrum, the transaction doesn't get larger if I have another 10,000 inputs that I'm not using in that transaction.What might be "fine" for you average PC/desktop, can really cause issues with hardware wallets as they struggle to process/verify all the data from 10s/100s of transactions etc.