I don't think so no. Apart form them being able to steal them there is the quite easy possibility of them losing it or being stolen for them
What are you going on about? Have you thought about the possibility that your seed phrase could also be stolen from you or you could very well lose it yourself without any help from your spouse.
If those are your only reasons on why your spouse should not know your seed phrase, then I think those are somewhat stupid reasons. A marriage where the trust or love levels isnt high enough to reveal your seed phrase to your spouse isnt a marriage I would want to be in.