Those secure elements may present other attack vectors, though.
And that my friend is the uncertainty that can keep a user up at night thinking about the possibilities. I absolutely trust Trezor's team to tell it like it is if they become aware of something. Trezor themselves may not even be aware of an eventual breakdown in the security of these new chips. Just saying!
What you have done is you have built a big wall around your house, making your home inaccessible and much harder to reach.
Not sure the metaphor you used is perfect but I get where you intended to go with it. I have examined the encryption from the SD's that I have been using for a few years now. The SEED and PIN are hopelessly encrypted without the SD present. To me, and this is just my .02, that is much better than merely placing a high wall around my house (Trezor).