You could always have it memorized.
This is really the worst idea in my opinion, who can remember more than 20 words containing uppercase and lowercase characters, it's not an easy thing to do especially for a long time.
You are right that it's not a good idea to memorize the seed phrase instead of writing it down, but take note that a seed phrase includes meaningful words and not uppercase and lowercase character. Also, it usually includes 12 words.