Yep, that's a good idea. It'll require a bit of change to my code actually, cause of the way its built (I'll spare the details, but the crawler bot is very modular and uses different HTTP libraries for different websites where required). I'll try to get on to that tomorrow. Thanks.