This is a funny scenario and I it obvious that you know that someone has your wallet phrase.
You can as well remove you fund from such wallet and create another wallet you can be using to keep your crypto coins.
If you don't mind leaving your coin there since it's just a small money, that could be okay but watch out in case you could not find your money in your wallet again, you can challenge the person.