When I send you money, I am only sending it to a Bitcoin address; I don't know your public key.
I know that.
Any public key that hashes to that Bitcoin address can spend the money.
But this is wrong. The public key is provided in the scriptSig, and that's what signatures are checked against. It's also how the hash is checked in the first place.
You can find a public key that collides with the hash, and pass the first part of a p2pkh script. If the hash passes, all that's left is OP_CHECKSIG, which still wouldn't pass with the dodgy key.
Bitcoin address balances are in the form of previous unspent transaction output (UXTO) payments they have received. A standard Bitcoin UXTO that would be in a wallet is a "pay to pubkey-hash", where money is sent to the RIPEMD160(SHA256()) hash of the public key (this
pubKeyHash is the same thing as a Bitcoin address, without the Base58 encoding making it pretty). The output script in this UXTO defines the procedures that must be met to spend the money.
A pay to pubkey-hash output script has instructions that basically say: if you can provide a pubkey, and the hash of that pubkey is the Bitcoin address included in the script, then you can spend the bitcoins with a message signed by the keypair.
The output script doesn't care that you use a particular pubkey out of the many that might exist that create the same pubkey-hash, only that the hash (address) matches. The spender is the one that is sending the pubkey to the network, saying in effect with their transaction "this is bitcoins I am allowed to spend - here I'm signing a message with the instructions to spend it, and since you don't have the pubkey yet to cryptographically verify my signature, I'm providing the one I used to sign the message".
Of course the signature has to be valid and the pubkey has to match the bitcoin address, but that is assumed.
The original generation of 50 BTCs from mining was a pay-to-pubkey script. To spend those, you had to have the correct key.