Do you think constant Firmware upgrades on hardware wallets are their weakness? I have gone through some firmware upgrades for some hardware wallets (Ledger) and I have to say for someone with good technical knowledge, it was not a good experience.
Yes I think it's a big weakness, especially if hardware wallet firmware is closed source like in case with ledger devices.
In this case you would need to fully trust developers to be honest, and won't make any mistakes that could allow hackers to steal your coins.
With open source wallets you can always verify the changes, and other developers can do the same reporting some potential issues on time.
The normal handling of the hardware wallet and the software is not that technical, but still a daunting task for people that are not that technical. (Thinking about the transition from the Ledger browser plugin for Chrome ..to the Ledger App)
I don't think ledger browser extension is working anymore, but they desktop app is also bad and having lot of issues with showing incorrect balances.
You can however use third party open source wallets like Electrum with ledger, to make things a bit easier.
What can be done to improve the firmware upgrade for these devices ...to make it "Idiot Proof" ? What happens when Ledger or any hardware manufacturer goes bankrupt and exploits are found and the developers are not there to plug the holes? (Export seed to Electrum?)
You can't do anything with black boxes like ledger, but you can change hardware wallet and get one that is open source like Passport, Bitbox, Keystone or Trezor.
Alternative option is to make your own DIY signing device like SeedSigner using general hardware like Raspberry PI Zero.