Some progress on the unconfirmed transaction (or mem-) pool

[This is one of the last BIG problems we had to solve..]

The previous post turned out not to be the solution, it simply wouldn't scale.

Happy to say we now 'really' have a working solution.

Trick was to construct the pool as a linked list and to use smart on demand sorting on addition.
This also solves our need that during block apply-ing valid and applied transactions can be removed very fast during iteration.
And allows for filtered iteration to kick out expired transactions.
And allows to put a max size on the pool so the expulsion algorithm can kick out transactions overflowing the maximum allowed (to mitigate DDOS attacks).

Finally we now have hooks to persist transactions to disk (have to do that ourselves since the database was removed).
And it supports transactional operations required to rollback in case of validation errors or to recover from system crashes.

The list itself, with active on-demand sort as you insert, has shown to do 12,000,000 operations a second (most likely double or triple that on a decent server).
This was tested with the default pool size of 600,000 transactions that fit in the HEAT mem-pool.
With real transactions this will become slightly slower because of the need to persist data to the memory mapped data store.

But the most important thing... the mem-pool will not be slower than any other part of the HEAT framework.
Which was the main goal.

For the nerds: HeatSortedLinkedList.java
[tip of the day! if you ever need to do a standard datastructure, look in the actual JVM source code for samples!]


I'm a bit behind on schedule because of the mentioned difficulties.
I'll get back to this later.