What I have done is - install the wallet, create the wallet and backup all sensitive data offline, then uninstall it. I believe this method is safe because I don't have a wallet on my phone - but when I need it I just need to install it [on the same device or another device].
To be honest, I don’t think what you suggested adds much security to your wallet. In fact, it might make it even more vulnerable.
If your phone is infected with malware, all it takes is for you to open your wallet on it once for the malware to steal your funds.
Also, every time you reinstall the app and restore your wallet, you’ll have to expose your wallet seed.
When you keep the app on your device, the wallet is stored on the device's internal storage, encrypted, which I believe is safer than having to type your wallet seed every time you need to make a transaction.
I might be wrong, though!