Who would be tracking Bitcoin users thru IP addresses?

I wouldn't be worrying so much about connecting home wifi directly without TOR or using VPN, everyone around the world I think already have been aware of BTC and people uses BTC every corner. Maybe the users involve in dark market especially if they are buying contraband.