Am I being overly paranoid?
My advice. Use an officially supported wallet. Choose a good passphrase, write it down and lock it away in a safe or perhaps give it to your lawyers for safekeeping (being sure to advise them not to copy or expose it). Backup your wallet and keep copies in several safe places. Your biggest risk is relying on your memory alone.
Aren't you also relying on memory for the passphrase to your wallet?
I don't have any lawyers, so I don't have that option for now.