Tell me about it... I've spent the last couple days watching youtube videos on the various AI tools available. It's hard to focus on one because new systems are coming out almost daily now. I saw a nice open source one that is meant to automatically scan the website and build an LLM off of it. It would need tweaking, since it cannot parse all off bct and will need historical training - but once it's up to date, the forum provides new posts to guests very easily, and the search engine can be just like Grok.
https://www.youtube.com/watch?v=JWfNLF_g_V0Yeah, it's insane how so many of these tools are popped off recently! These are all wrappers over the free LLM's offered by the likes of Facebook/Google/others. We wouldn't go for a wrapper since no one would want to pay them monthly and also it's not so fun! We would need to create a crawler that would either parse the whole bitcointalk or just train one of those FOSS models on data provided by LoyceV/Theymos/Bitcointalk.
I think LoyceV can provide you will all the original forum posts. I'm not going to do any coding because by the time I learn how to do something, I discover there is an AI bot that will do it in seconds. :/ In fact, I'm pulling away from most parsing projects to instead focus on things I enjoy. I can be an idea guy if you ever need a different perspective of something.
To give a different perspective as someone who writes code for 8+ hours a day, AI actually sucks at coding! Fine, ChatGpt/Claude can create a CRUD APP or a React Component with some boilerplate code in seconds but real-world apps are much more complicated than that and even complicated if you're working with event-driven architectures, payment gateways, etc. It's knowledge and reasoning are very limited and at very best you can use AI as an assistant to write repetitive or boilerplate code. So yeah don't hesitate to learn programming!
I assume you have resources available for development? Most cloud companie give free trials. If it launches and is popular, you'll obviously need to charge per search, or have it forum sponsored.
Don't think I can run any of those models on my laptop...I can have a chat with couple of my Data Engineer friends and see what they recommend!