Post
Topic
Board Development & Technical Discussion
Merits 18 from 3 users
Re: World's fastest and simplest block parser for those who (only) need all HASH160
by
BTCW
on 05/03/2024, 20:23:26 UTC
⭐ Merited by LoyceV (12) ,ABCbits (5) ,vapourminer (1)
For those interested in mempool analysis (I am), I created a new "bash pipe monster" that isn't pretty but gets the job done.

First, in Bitcoin Core (fully synced and indexed), simply "savemempool", and it will spit out a file called mempool.dat

Along the lines, pun intended, of the OP, this will make it human readable:

Code:
cat mempool.dat | xxd -p | tr -d '\n' | tr -d '\r'| sed 's/0100000000000000[0-9a-fA-F]\{4\}0000000000000100000000010/0100000000010/g' | sed 's/00000000000000000\([1-2]\)000000/\n0\1000000/g' | sed 's/0\{9,30\}\n/00000000\n/g' > mempool.hex

It should work on both Windows and *nix. The output, mempool.hex, will be a text file of all mempool transactions (a snapshot according to your node), in which every row is a Bitcoin raw transaction in serialized hexadecimal code.

It's not 100% perfect; some rows are empty, but ~99% of the rows are transactions that https://live.blockcypher.com/btc/decodetx/ think are OK and/or have already been broadcast.

My last round gave me ~42k candidate transactions (one per row) and the size of mempool.dat was close to 100 MB and the size of the resulting mempool.hex almost double.

It is good enough for my purposes (currently: signature analysis). Do with it as you wish! (Don't ask me to explain the regex; it took some severe trial and error. It helped when I realized 16 zeros in a row are a separator between transactions [undocumented], and it allowed me to insert a newline character at the right spot.)