Someone owed me money and he wanted to pay me in Bitcoin.
So I gave him my Bitcoin wallet address and he transferred the funds.
It's now complete.
But can he now withdraw the money since he has my bitcoin wallet address?
No, that money can’t be withdrawn, having your bitcoin wallet address does not mean they have access to your account. Only when you give out your private keys or seed phrase will the sender have access to your wallet. That is why it is necessary to keep what is private a private entity to your self and not disclose to anyone or the public. The seed phrase functions the same as the private key of the wallet.