Trezor uses two-factor authentication - something you have (trezor) and something you know (PIN). If an attacker has physical access to your Trezor and he also controls your computer, you're screwed.
Not wanting to be negative or anything - Trezor is definitely a great improvement on security since our greatest worry right now are malwares - but we should note that its currently configuration makes it vulnerable to the "evil (and tech-savvy) maid attack".
Somebody tech-savvy enough with physical access to your place (a maid, a boyfriend/girlfriend, a roommate etc) could deliberately infect your computer while you're not around to get the PIN. Even a hardware key-logger would suffice. Once with the PIN, s/he only needs to steal the device.
I don't think that's a major risk, in the sense that it won't happen frequently, but it's worth noting anyway. Some people can't afford to fully trust those they share their living space with. Think college students for ex., particularly computer science college students who happen to share their room with people who were just put there by the residency administration. They barely know each other...