This has always confused me. If you buy coins from an exchange, won't the exchange link your Bitcoin address with your ID and therefore track every transaction thereafter? In effect, can't exchanges completely de-anonymise transactions if it starts from them?
There are differences in exchanges. For example, if you use Coinbase which is very legal and legitimate they will ask for all your identity information. If they suspect any connection to anything illegal with your account you are at risk of account being disabled and your info turned over to authorities, even if you didn't do anything, maybe you were just curious and visited some sites that sold illegal goods.
Alternatively you can use an established exchange like BTC-e.com. They are one of the oldest Bitcoin exchanges, but since they operate anonymously (the govt doesn't really know who they are) they can relax rules about how much info they collect and what they by law must track or report.
If you buy in person of course that's a different story. You don't give any identification, but you should still be careful not to get robbed. Use a very public place and meet inside, like a Starbucks so if you yell Help the person can't easily escape. There was a guy that met in a public place, but it was outside so he got robbed and the thief just ran away.