On the other hand, the day the IRS can tell me how much wealth I have in my (vastly larger quantity) brain wallet is the day I pay taxes on my true wealth.
Spoiler: This day will never come.
Unless you mined those coins in your brain wallet yourself, the IRS doesn't really care what you have in it until you convert it back to fiat. That's when you calculate your capital gains.
Tell you a secret...
By the time I would actually need to convert my brain wallet wealth back to fiat, there will be no fiat left to convert back to.
The IRS will soon be as irrelevant as Blockbuster Video, and there is nothing anyone can do to change that.