The worst thing can happen is your balance will become zero, nothing else. You no need to worry about anything else cause all the transactions are already public and one who have the seeds may not connect with your identity so their only intention will be taking the money in the wallet then just let it be.