If you want to reach your wallet you have to write your private keys or open your json file etc. If you will do one of these with scam wallet yes you can loose your tokens immediately.
I think you can use mew mycrypto or trust wallet for now for ethereum and please dont try any other new wallet. These times there are too many scammers and we can not trust anybody.
If you are giving your private key someone or someplace they can steal your tokens easily.  But only if you want to use that service for like etherscan and only for tracking your tokens in your wallet without entering your private key they can not steal your money.