First.. I apologize to everyone else (but the troll) for my anger,  I shouldn't let trolls create anger, its what they want.

I'm guessing the way that could be linked is when you're using clearnet and you can see your IP on irc,  could be linked through what's called "coincidental occurrence"
ie. watching what IP is active at certain times when transactions happen,  plus if they are a seller they could guesstimate a link between IP and the transaction.

This is just my best guess of the process someone might use.