Except one leaves his wallet security details open on the internet or to a third party they will not have access to those coins, that's why we must keep our private keys safe, seed phrase as well is not what should be tempered with, someone can't track you down and steal your funds with just the help of public keys, but your seed phrase can make them discover and acquire your coins if left vulnerable.


It's not that easy, no one can steal your coins with just the use of public keys, though they can view your account on the blockchain since it's an open distributed ledger but can't alter it, that's why the bitcoin network is secured and trusted.