Quote from: ltcltcltc on Today at 08:37:08 AM

So, please tell me if I'm wrong: you just scrape content with limited data treatment, so the filtering functionalities you offer are the same that the forum offers, i.e. sorting by chronological order, viewing the posts inside a topic and filtering by user.

I don't process anything, I just keep the raw HTML for archiving purposes. Although I also keep a list per user and per topic.

Quote

Ok! I thought by the previous message that you were declining.

I meant ask TryNinja nicely if you want for instance only data from the Economics board.

Quote

But if that's not the case then I'd be super grateful if you shared your database with me to avoid the undesirable task of rescraping the scraped!

I don't have a database. I just have "data". And it's a lot. Hence my question if you know how you're going to handle it. Old posts for instance are stored in a different format, although I may still have a backup of individual files for each post. That's the part where you'll get millions of files in one directory. So you'll have to be a bit more specific before I just dump a shitload of files on you Tongue

Quote

Ninja's website looks handy too. Harder to scrape though.

Don't scrape, ask Tongue