What you describe is indeed blackmail.
"If you do/don't do , I will " is the blackmail formula.
I'm not an expert on blackmail, but I'm pretty sure the above in certain contexts could be considered a warning instead of blackmail. If a police officer tells you to stop speeding, or else you're going to suffer the consequences, that's a warning. He isn't looking to get something out of the transaction personally--and I think that's a key component of blackmail/extortion.
If someone sends a PM to a member on bitcointalk stating that if they don't stop spamming in a thread that they're going to get reported/negged, I would see that more as a chance to correct behavior before the offending party suffers some consequences. It isn't threatening to expose someone in exchange for money/favors/whatever, which is usually what blackmail is.
I am afraid to send PM to anyone even when found spamming because I saw that this can leads to temporary ban for PM spamming.