I just realized you're not showing create/edit timestamps for posts. Do you have them? I can give you that too.
I don't. That would also be great. Can you send me only the date they were posted? (no edit timestamps, etc...).
I tried searching for invalid (non-numeric) topic ID and it seems to spin forever. If combined with user it seems to return all posts for that user.
I'll fix that when I finish changing everything to search the new database layer (more below).
I assume you're scraping the page with the post to check for updates, right? That means you're downloading up to 19 other posts at the same time. If you download them anyway, can you add those posts to your "check for editing" feature?
Yes! You are absolutely right. I'll do that.
I found out that updating a lot of data in a database full of indices is a pain the ass. The database needs to re-create every index, which takes a lot of time (mainly due to the reverse index that allows the "content" search).
I'm currently setting up a second database (using Elasticsearch) which will make searching and data analysing a shit-ton easier. Because of that, the "search post" page will be unavailable for a bit in a few hours.