With your wallet address, no one can withdraw money from your wallet. The only way a person can withdraw money from your wallet if you are using an exchange is when the person has the mobile number or gmail that you used to register on the exchange and the password to the exchange. If you are using a non-custodial wallet, then until the person has access to your private key before they can make a withdrawal from your wallet, that’s why we are supposed to properly secure our private key so that no one will be able to access it. But with your wallet's private key properly secured, your bitcoin is safe.