Even if your Trezor works as it is supposed to, you will still be vulnerable to attacks like address phishing (the hacker tricks you into sending payment to the wrong address) and man-in-the-middle (a compromised PC software displays the correct destination address on the screen, but puts the thief's address in the transaction that it gives Trezor to sign. While an alert user can notice the substitution by checking the Trezor's display, there will inevitably be users who check only the PC screen, out of laziness or because they are not aware of the risk.)
I'm not sure if Trezor supports it already, but shouldn't the payment protocol solve that particular problem?