I'm actually sad that the Silk Road guys did drugs first, since all it did was bring a bunch of heat down.
The problem is that that excuse can be used for anything. The obvious solution is to just use silkroad.
Using a marketplace, even if other people do illegal things there, does not make you a criminal. I buy things downtown all the time, some people buy drugs there, if proximity to crime was a crime, you are already guilty, get over it.
For the hyper paranoid: If buying/selling gold becomes criminal, then at least you are doing business somewhere that takes security seriously.