Have the criminals corrupt themselves.
As they say: Evil will do upon itself and eradicate the wicked from all of this earth.

That would just worsen the scenario. If they are corrupting themselves, they will soon corrupt the system and all will do the same and will result to much worst. Better to stop them while they are doing it and eradicate them. But that's true, if they kill each other because of it, that's great, which is highly unlikely to happen, as they would just connive with each other to do a much bigger conspiracy to corrupt more.