How about deny everything and have people enter their worker IPs when they setup workers. Then just open up the source IP for each worker.
Although this would solve problems for the pool servers, we "the masses" are not very good at doing this. I'm sure most people who are mining right now know their way around an IP address. But the "average" person doesn't. Plus then you have to deal with dynamic IPs and the whole thing gets way to complicated.