This still leaves your coins vulnerable because if someone gets access to your phone, they can steal them. I'm talking someone hacks their way into your phone in some manner. Not physically grabbing it from your hands.
Your idea may work if your phone didn't have service and when you wanted to transfer your coins you would turn on the service and transfer the coins.
I'm looking for 100% way to pull my bitcoins from internet/computer and put them onto a USB drive so only way to get access to them is to have the USB drive.