Someone owed me money and he wanted to pay me in Bitcoin.
So I gave him my Bitcoin wallet address and he transferred the funds.
It's now complete.
But can he now withdraw the money since he has my bitcoin wallet address?
Great to have a buddy that returns what's due

But unfortunately if you are dealing with a person with the technical know how of crypto, worst case scenario here is when he/she sends the coins and acknowledge receiving your payment , then the worst they can do is double spend to try and not get those coins to you otherwise get the coins forcefully from your wallet is in possible..unless they have access to your private key(s)..
Otherwise nothing to worry about buddy, enjoy your coins and remember not your keys, not your coins!