I don't see botnets as an issue, if they become too often there are ways to prevent them for pools (and if they begin using there own pools there could always be changes in the bitcoin client to prevent this).

What is killing mining for an average user is the hardcore users (obviously). I mean, the output from an average user in BTC no matter its worth is so ridicously low compared to someone that will own a whole mining server.