There is no chance for them to track you as long as you are using decentralised wallet but if you want to choose centralized then that is voluntary act and it is you who give an access to them  to track you. But I think there is no reason to be bother because our technology as time passing by is becoming more advance so there will be a system that will protect the crypto holders to make their identity confidential.