it is kinda worrying leaving your seeds somewhere someone can find but for sure there are solutions. This is the real risk of crypto currencies.
If you re worrying about your seed phrase and you do not want anyone to steal your coins, you can add passphrase to it. If you add passphrase, you are safe but you have to let the passphrase be strong. Eight to ten or more characters that consists of alphabets, numbers and characters like .,@ should be enough.