Think more along the lines of a small custom device with a screen, a couple of buttons, and a serial port (or serial over USB, or serial over bluetooth, or serial over NFC, etc). The programming interface, if it has one, must be internal, or it must load software from a memory card (like SD).
What about SIM cards?
I wouldn't trust a device without a display and keypad (or at least a pair of buttons!) built-in.
I also dislike the idea of a device just signing a transaction presented to it, rather than generating the entire transaction internally. But, this can be overcome.
Also, in my opinion, the device
must generate all keys after it enters my exclusive possession, and it must include a genuine entropy source. This may be extra paranoid on my part, but I'm not sure I even like the idea of importing keys into the device.