Post
Topic
Board Development & Technical Discussion
Re: PointsBuilder - fast CPU range points generator
by
farou9
on 30/04/2025, 02:49:51 UTC
So they are all stored in 1 file ?

And how do I run it ,i can deal with c++ and py but c I have no idea how that works

Compile it and run it according to the README maybe. You can do what you want with the database afterwards, as long as you interpret the values as offsets to the base key of the range.

The code seems stable, looks like the only cases where results are wrong if you give it a range start that equals 512 (or N - 1534). I'll add some checks for these two edge cases. So you're good if you want to dump keys from 1 to 1.000.000.000 Smiley But you have to disable the check in main.c to do that, for now.

As for your first question: yes it dumps data into a database. I have a feeling you want something that can dump 1 billion points to disk instantly. However this code streams keys in non-sequential order, because of how the ranges are scanned in parallel.

I'm not aware of anything faster than an actual database if you really want the X values indexed in order to be able to later query it to get the scalar. The only thing that's faster is to not use a database and work in RAM.
But the question is what is structure of the database if its 1 file there is no advantage for lookup .
So they are stored randomly.

Well that much ram is expensive.