I'm not sure where you'd put it. I think most suggest pastebin for plain text and I assume it's raw html that you have gathered.
Lol no, I can't paste 1.5 GB into pastebin

Is there a script you have used to gather this?
Basically, I
wget "https://bitcointalk.org/index.php?action=recent;patrol" every 5 minutes.
I don't really want to kill the forum's server by sending pings every 5 minutes.
You're allowed 1 page per second, 1 per 5 minutes isn't going to kill the server. I do more than that just browsing.
I did have a script that attempted to clone every page on this forum directly from the forum's server (starting with 1) but it didn't incorporate the 1 second request limit so it just kept on getting a 502/503 server too busy page
A one second delay between requests fixes that. But, with more than 30 million posts, that will take at least a year. A huge waste of resources if you ask me.
Did you report their posts? They do nicely fit this thread:
Great project!
Good project
