Any issues with this approach also exist with your device. Your device just contains even more issues due to its closed source code, which I assume is a backdoor only reason it is not in the open is it contains malicious secrets. The difficulty of trezor shipping some modified hardware in mass scale and not being detected when the hardware itself is open source design is nil given that the hardware is more easy to audit (and less than yours given you don't even have open source hardware and your hardware is harder to audit). The problem is your device contains all the same risks as trezors device PLUS the additional risk of the closed source software.