How about trying to memorize it? Is it possible?
You're talking about memorizing 12 words, not that hard!
What are the benefits? ... and the setbacks.
The benefits are that you will not have to access your secret stash where you keep your private key every time you need to restore your wallet nor that you would need to carry that thing with you when you travel or you move temporarily!
Setbacks there are none unless you talk in your sleep and you sleep among strangers maybe?
So as long as you have that sed also on a physical back-up stored safely there is nothing wrong with trying to memorize it.