Post
Topic
Board Development & Technical Discussion
Merits 6 from 1 user
Re: Are there any benchmark about Bitcoin full node client resource usage?
by
gmaxwell
on 09/05/2021, 14:44:55 UTC
⭐ Merited by Welsh (6)
I'm afraid, this PR wasn't helpful enough for utilizing modern multicore CPUs:
[...]  In practice, it has been experimentally shown (and even discussed in the PR 2060 related comments) that the overhead of the scheme rapidly outperforms gains once it is configured for utilizing more threads than the actual CPU cores and it stays around 30%

Of course something stops getting more gains if it uses more threads than CPU cores.  That PR is from 2012-- I linked it to show just how long tx processing has been in parallel, there have been subsequent changes that further improved the performance further.

Quote
Firstly it is NOT about parallel transaction verification, instead it postpones script verification by queuing it for a future multi thread processing which is called right before ending the block verification that is done single threaded. This thread joins to the final multi-thread script processing.
The logic behind such a sophisticated scheme apparently has something to do with the complexities produced by transaction packages where we have transactions with fresh inputs that are outputs of other transactions in the same block.

Stop engaging in abusive technobabbling.  There is no interaction with "transaction packages" or anything of the sort there.

The verification of transactions runs in parallel with everything else. One thread loads transactions from the block into a queue of transactions that need to be validated, other threads pull transactions from the queue and validate them.  When the main thread is done loading the queue, it too joins into the validation which has been in progress the whole time.  There is nothing particularly fancy about this.

Quote
Secondly, it uses cs_main lock excessively, keeping the lock constantly active through the script verification itself, hence leaving little room for parallelism to have a scalable impact on the block verification process.
All you're doing is copying from the PR summary, but it doesn't even sound like you understand what the words mean.  What the PR is saying is that it holds cs_main until the block finished validating, so that the user never sees a incompletely validated block.  When blocks contain only a few inputs this limits parallelism, but once blocks have a couple times more inputs than you have cores it achieves full parallelism.  Remember again, this was written in 2012 when most of the chain had only a few transactions per block, so the fact that it was parallel for only a block at a time was a limitation then-- it's not much of a limitation now.  Today a typical block contains over 6000 inputs, so there is ample parallelism even when limited to a single block at a time.

Of course, there is always room for improvement.  But one improves things by actually understanding them, writing code, and testing it.  Not by slinging verbal abuse on a forum in a desperate attempt to impress the peanut gallery, so I guess you wouldn't know anything about that. Cheesy