Common smartphone users don't actually do this
Absolutely. But common smartphone users all do a bunch of other widely insecure things, such as using biometrics, keeping their 2FA app on the same device which has all their passwords saved, installing a bunch of apps which track everything they do, using terrible closed source wallets and then storing significant amounts of coins on them, back up sensitive information to cloud storage, and so on. And even among people who use good open source wallets, very few of them actually properly verify those wallets, and even fewer of them have ever actually looked at the code or tried to build the wallet themselves.
But the question wasn't "What do people commonly do?". The question was how to ensure that the app you are installing is doing what you think it is doing. The answer to that is as I described - download the code, review it personally, then build the app yourself from that code. Unfortunately lots of people take risky shortcuts and often end up paying the price for doing so.