This is the base concept behind asymmetrical cryptography (https://en.wikipedia.org/wiki/Public-key_cryptography).
You can share your public key with anyone, so that one can encrypt a message that only you can read (owner of the private key derived from the public key).

When it comes to bitcoin (or more generally to cryptocurrencies), your private key allows you to spend the coins you received on your public address (wallet address). (More info here : https://en.bitcoin.it/wiki/Technical_background_of_version_1_Bitcoin_addresses)