You are just selling BTC to anyone interested. You are not an investigator on the people you sold BTC to. Just like a money exchanger accepting clients without necessarily investigating whether they are going to use it for crimes or not. How would you know?
I forgot to mention that BTC itself is illegal in our country which is the major reason of my fear. I sometimes sold BTC to unknown guy and that's what I'm fearing of. If it was legal, I would have nothing to fear at all.