Lets get things straight.

It's an issue on localhost, for each and every getwork request pushpoold gets - it opens up a NEW socket against bitcoind, and issues getwork over that socket.
When it's finished and pushpoold got his work, the socket is left untouched and still opened.

We (and I know BTC Guild have) tweaked Linux a bit to reuse TIME_WAIT connections (using tcp_tw_recycle in /proc/...) which has helped - but it's not scalable.
As the load increases, the amount of new connections is overwhelming.

This was taken yesterday by me:

At that moment, we had 21.000+ TIME_WAIT sockets ready to be reused and/or closed due to timeout.
When that figure reaches 25k+ - the server starts to get hard to keep up, we're also hitting limits with num openfiles (nofiles in ulimit) which we increased from 1024 to 128.000+.

We're looking for a solution to get rid of "all" those TIME_WAIT connections (which 99.99999% comes from 127.0.0.1:8332 (bitcoind) against pushpoold at 127.0.0.01)
The best way I've come up with is implementing keep-alive support in bitcoind - hence this thread.

This will mean that bitcoind and pushpoold are only using a couple (multi threaded) keep-alive sockets for getwork and sending responses.
This will drastically lower the amount of open sockets (nofiles dropping to the bottom) and free both ports (limit of 65554 ports in TCP/IP afaik) and also make it more scalable.

Are everyone following now?