Really looking forward to the Tech and how it compares to Ardors proposal.
Using memory mapped files instead of DB will allow much, much higher transaction capacity.
Not just using memory mapped files (which are amazingly fast).
Also the structure of the data is important, where no indexes are needed for data only accessed sequentially, there are no indexes.
The memory mapped files are so fast actually that at almost no extra cost we can write both the "real" data and write "recovery" data before that.
This is how we implemented atomic writes where no matter when the system crashes, our data storage can always recover on start up.
But don't be fooled that we have a poor API or lack of deep data access since there is no embedded database.
Through a method we (
and others) call
replication we can keep a MySQL db (or a cluster of them) in perfect real-time sync, which allows us to build a web frontend that shows live transaction data. But also allows for real-time instant order matching against existing orders in the db.
We have already used the replication mechanism (event based mysql sync) in a commercial project Eliphaz will undoubtedly tell more about in the near future.