Another very good argument is that somebody in the house would do exactly that too (put a SIM in) because of not knowing it's a "special use" phone.
Another argument is that a burglar is unlikely to expect you to use a smartphone for a cold storage. If he sees a phone to steal, the steps he'll do later are pretty much known. Reset, and sold to some stranger for a ridiculously low price. Also, as said, you can use a password to encrypt everything.
The best argument for not using a smartphone as cold storage, is that desktop wallet software such as Bitcoin Core and Electrum are simply more tested, and are consisted of a bigger community. Another good argument, is that a linux distro is far more tested than a smartphone OS.