You can't just access a wallet if you know it's address. You need to know the private keys, which are private.
Once you have even one single outgoing transaction, your public key is NOT private.
Right, the public key is never supposed to be private. The person you quoted is referring to
keys.