Quote from: LoyceV on January 04, 2020, 08:44:07 AM

Quote from: PrimeNumber7 on January 03, 2020, 07:08:15 PM

You can direct your script to review multiple pages of recent if it detects that all the posts it most recently scrapes are new, it can continue doing this until it reaches the last maximum recent page the forum provides.

I've thought about it, but haven't had the time to add it yet.
Ideally, I should switch to scraping from another server if I miss a few posts. That way, I could even cover downtime if I reboot the server. But that's for later

If you want, you can send me your code for your scraper. Adding a counter and telling your script to check the next page is something that should be very simple (< 5 mins, including testing). If I am familiar with the programming language (or if I can read the documentation necessary to help in a short time), I can give you suggested lines of code to add.