I may not be perfect but I know well the calibre of the kind of people I walk with, you can't expect me to be around the same kind of positive thinking minds and yet be afraid, also, I don't think it's wise for you to allow people around you to specifically in now what am upto.
Aren't you going to give the bad guy your wallet address, and you let him get killed. However, your Bitcoins remain safe because you have given your wallet address to someone you trust.
Or you will give it away because you still want to live, and you don't have any more Bitcoins.
At this point one have to be wise, if you try any shit with these guys they may end up killing you, so it's better they even get access to the wallet and later after the whole show, you're being alive and can get another bitcoin with time