A fake or scam app can gain access to your phone, but it will never know your passphrase.
Which of course should only be stored in your brain and not on any devices.
You can write it down somewhere safe as a physical backup in case your memory should ever fail you.
That's only works if the users didn't install on their phone, in real life I see most people install their wallet because it gives them convenience to check the balance/incoming transactions rather than check through explorers.
This is why I always prefer bookmarking the wallet's real site and find the download button in there that redirects me to the app store.
That's only make sure you're not fallen to phishing site, but it doesn't verify whether the wallet is legit or not. Hence you need to verify the PGP/GPG key.