If you Know your family well then i don't think this will ever takes place not unless you grown from different places so you know nothing about the family members then they might have that Idea of taking your funds out.
but as long as the people you are sharing are someone that you know well? I am sure they will never take your funds(but still need to stay silence about your holding) because like me, Only Wife and my parents have idea how much I am Holding now because they are the only person i trusted the most.
When it comes to money is really hard to trust someone even if the person is very close to you, like you said, is your wife and family who knows how much you're holding. I believe you can as well say out your seed phrase to them right?
All I'm saying is money has a way of changing people, in some cases if they know how much you're holding they might try to know your phrase and even try to get hold of your funds, things might get messy.
Since greed works in people, it takes no time to change. Especially money-related matters can cause a quick reaction, even a trusted person in one's family can do this. I don't think it's better to tell about the money. And if someone needs to be informed then only he can be informed that he has money but to reveal how much money he has can create a different reaction. Even if you don't trust an outsider, you must trust a member of the family. But you have to take responsibility for your own financial security or else it will be risky. As much security as possible regarding crypto assets that will be better for the assets holder.