It's like having a safe deposit box at a bank. Your private key is the key to the box, the public key is the box number. simplyfied