No harm in hoping.
I just want to first see how fast can it solve smaller bits 70 down
OK. I cooked up something that uses all cores to build all the points in a given range.
It can do 50 - 60 MK/s, if you have 64 cores. But the problem is the saving - SQLite can only ever write in synchronized mode (which makes sense, since it needs to check the index), and the best I got is around 300.000 inserts per second. Saving to files is a non-sense because then you won't be able to efficiently query the X value (which is the main point after all).
I'll upload the code tomorrow on GitHub so you can test it.
Meanwhile, the Scooby Doo code seems to be under scrutiny.
The only comment I have is that when something claims to work better than something else, it is irrelevant what / how / why the "something else" works. Introducing any kind of relationship or contract between the ways things operate has nothing to do with common sense. Maybe the next Scooby Doo version will be a server-side API call that finds a key in a range. If someone can find a method that works better than something that is totally opaque and that has no prior knowledge on the inner guts of how it works, only then can there ever be a claim of a method that works better than the
Scooby Doo method!