I really love the idea of a brain wallet. So, if your computer is destroyed, the only evidence the bitcoins belong to the address you had in your wallet is through the blockchain?
That's not the point of a brain wallet.
So, you don't really need a physical location for your bitcoins?
Exactly. It's meant as a way to 'store' a wallet without depending on any physical storage or online wallet service.
There is no evidence in the blockchain about who owns what addresses. The only thing you can detect is if two addresses appear on the input side of a transaction together, which is an indication they belonged to the same wallet (and thus to the same person). Although you still wouldn't have any clue who that person is.
If you want to avoid evidence that a certain address or wallet belongs to you, just store it on an encrypted drive or file container (e.g. TrueCrypt). Which is a good thing to do anyway, also because of theft (if someone steals your laptop and the wallet is encrypted, they can't do anything with it).