Re: Some 'technical commentary' about Core code esp. hardware utilisation
Quote
And many people on the project quit because they didn't like working with you, what's your point?
Name one.
Quote
People have been laughing at your choices for years and here you are defending it because you wrote some codec to watch porn with higher fps some years ago.
Says the few days old account...Quote
Inefficient data storage Oh please. Cargo cult bullshit at its worst. Do you even know what leveldb is used for in Bitcoin? What reason do you believe that $BUZZWORD_PACKAGE_DEJURE is any better for that? Did it occur to you that perhaps people have already benchmarked other options? Rocks has a lot of feature set which is completely irrelevant for our very narrow use of leveldb-- I see in your other posts that you're going on about superior compression in rocksdb: Guess what: we disable compression and rip out out of leveldb, because it HURTS PERFORMANCE for our use case. It turns out that cryptographic hashes are not very compressible.
Everyone knows compression costs performance, it's for space efficiency, wtf are you even on about.
Most people's CPU is running idle most of the time, and SSD is still expensive.
So just use RocksDB, or just toss in a lz4 lib, add an option in the config and let people with a decent CPU to enable compression and save 20+G.
Reading failure on your part. The blocks are not in a database. Doing so would be very bad for performance. The chainstate is not meaningfully compressible beyond key sharing (and if it were, who would care, it's 2GBish). The chainstate is small and entirely about performance. In fact we just made it 10% larger or so in order to create a 25%-ish initial sync speedup.
If you care about how much space the blocks are using, turn on pruning and you'll save 140GB. LZ4 is a really inefficient way to compress blocks-- it mostly just exploits repeated pubkeys from address reuse
the compact serilization we have better (28% reduction) but it's not clear if its worth the slowdown, especially since you can just prune and save a lot more.Especially since if what you want is generic compression of block files you can simply use a filesystem that implements it... and it will helpfully compress all your other data, logs, etc.
Quote
So what's your excuse for not making use of SSE/AVX/AVX2 and the Intel SHA extension? Aesthetics? Portability? Pfft.
There was an incomplete PR for that, it was something like a 5% performance difference for initial sync at the time; it would be somewhat more now due to other optimizations. Instead we spent more time eliminating redundant sha256 operations in the codebase, which got a lot more speed up then this final bit of optimization will. It's used in the fibre codebase without autodetection. Please feel free to finish up the autodetection for it. It's a perfect project for a new contributor. We also have a new AMD host so that x86_64 sha2 extensions can be tested on it.