Apparently, there is a vector attack where the address in the Wallet UI is the user's one, and the one on the HW wallet is the attacker's one. So use extreme caution.
Wait.... are you saying there are people who use a hardware wallet but don't confirm the address on the screen? That's one of the main reasons to use a hardware wallet: to verify you actually sign a transaction to the correct address!
I'd be much more concerned if an address changes
whenbefore I
readsee it on my screen. Let's say you deposit to an exchange, but the deposit address on your screen is changed already before you copy it. That would mean the address on the hardware wallet matches the address on your monitor, but it's
nownot the exchange's address.
I have no idea how (un)likely this scenario is, but it's always on the back of my mind when making a transaction. And it's impossible to verify.