Post
Topic
Board Development & Technical Discussion
Re: Are there any benchmark about Bitcoin full node client resource usage?
by
gmaxwell
on 09/05/2021, 21:41:25 UTC
Besides the fact that throughout the source code, in the comments, transactions with dependent inputs are referenced using this term, I tried but didn't find any other term to use for such "packaged" transactions when they are present in a block, it'll be highly appreciated if you would give me one.

100% of the use of the word is in the context of the mempool and tx selection.  As mentioned 'packages' were introduced as part of child-pays-for-parent mining.  Blocks have always contained transactions whos parents were in the same block from day one, the package concept in the mempool was introduced so that it could include high feerate children when the parents wouldn't otherwise make it into the blocks.

There are simply no packages in blocks or in block validation.  The term you want is just *transaction*, maybe child transactions, or "transactions with unconfirmed parents".  There really isn't a term for what you're looking for because supporting the case of child txn in the same blocks just works in Bitcoin with no special additional code, so it's never been a separate freestanding thing that needed a name.

Calling that packages would be inappropriate because it suggests they're in some way grouped or something-- but they're not.  A dependent transaction can be anywhere in the block so long as it's after its parents.

Quote

So, it is not about parallel transaction processing because other than their input scripts, transaction are processed sequentially.
The validation of a transaction is almost exclusively the validation of its inputs.  Outputs are only checked for size and total amount, nlocktime is an integer comparison, as is the maximum weight. For these kinds of checks bouncing the cacheline over to another thread would take more time than the check. 

It could queue the entire transactions rather than the individual inputs but this would achieve worse parallelism e.g. because a block pretty reliably will have >6000 inputs these days, but may have anywhere between 500 and 2000 txn just the same.  At least at the time that was written working tx at a time was slower per my chat logs. That might not be true anymore because more of the workload is full blocks, because more recent cpus suffer more from overhead, etc. This was the sort of thing I was referring to in my prior post when I said that less parallelism might turn out to be faster now, due to overheads.

Quote
Few weeks ago someone in this forum told something about block verification in bitcoin being sequential because of "packaged transactions" (take it easy).
Turns out that people around here often have mistaken beliefs.  Among other sources they get propagated by altcoin scammers that claim to have fixed "problems" that don't even exist.  But just because someone was mistaken about something that is no reason to go on at length repeating the error, especially when someone shows up, with citations, to show otherwise.

The validation is multi-threaded. Period.  Stop claiming that the validation is single threaded simply because it's not inconceivable that it could be made faster or more parallel.

Quote
I proposed an algorithm for this and the discussion went on in other directions ...
Your proposed algorithm would almost certainly not work well in practice:  Every one iterations requires multiple read/write accesses to global data (the set of existing outputs).  Also in the worst case it would have to attempt to validate all transactions N^2 times: consider what happens if all transactions are dependent on the transaction before them and the parallel selection is unlucky so that it attempts the first transaction last.  It would process and fail every transaction except the first, then process every transaction except the first and fail every one except the second and so on. More commonly each thread would just end up spending all its time waiting for control of the cacheline it needs to check for its inputs.

It's just trying to impose too strong an ordering requirement, as there is no need to validate in order. Bitcoin core doesn't today. Deserialization must be in-order because the encoding makes anything else effectively impossible. And the consensus rules require that outputs exist before they're consumed but that can be (and is) satisfied without ordering validation by first gathering up all the new outputs before resolving the inputs from txn that might consume them.

Quote
Additionally, I noted the excessive use of locks and became concerned about their impact on parallelism.
What use of locks did you note?  Search in the browser in that thread shows the word lock never comes up.  And during the input validation the only mutable data that gets accessed is the signature cache which has been lock free since 2016, so the only lock the validation threads should even wait on is the one protecting their workqueue.

Quote
Honestly, this discussion didn't add anything to my knowledge about the code and bitcoin
Doesn't seem that much of anything ever does...