Not necessarily, if at the time of creating the wallet on your smartphone you copied the seed on a sheet of paper, you can restore it on a new device. The only problem you can have is that someone steals your mobile and manage to guess the wallet password, but if you react quickly, restore the wallet and send the funds to another different wallet of yours, you will not lose your coins.