The more you ignore them the more you would become vulnerable to their hate. It is better to find their weakness and then downgrade them infront of everyone. In today's time everyone has some kind of addiction. You can use that as a weapon to defend yourself rather ignoring them. No one is a saint in these times.
.You are correct; we should always target their flaws and exploit them. We are humans, and our actions are frequently both beneficial and detrimental. Obviously, we are not saints who always do the right thing; rather, we should address what causes our faults and find a method to correct them right away. People around us will always look for our problems, so we must either react or ignore them. In my opinion, I would just choose to ignore them since I don't have time for problems, and I have more useful things to accomplish with my precious time.
A common reaction when you are in such a situation, finding things to throw back to those people who are judging and like what being mentioned we are in the generation where everything can be exploited and there's always venue that you can use to pull your own trigger, though I like your position in terms of letting them to continue bringing the problem on their own and use your time to something that will benefit you.
Instead of doing the counter, just leave it as it is and let them enjoy with whatever they are thinking about you..