Hrm. I would have answered, "how do I know a man in the middle didn't give me their public key instead?" But the same weakness exists with CAs. How do I know a man in the middle didn't add their key as a CA when I downloaded my browser?

It's tricky.

...I wonder if people still use Namecoin.