test

Right, it's 18 ZB, roughly the same as 24 ZB (using 20 bytes per address, i.e. just HASH160, no prefix, checksum or private key).
In other words, about twice as much as all the world's storage capacity (HDD, flash, tape, optical) in 2023, and would cost something like $1000 billion¹.


¹ Using data from IDC and their "Worldwide Global StorageSphere" metric (not to be confused with the "DataSphere", which is the amount created and some 10x bigger).

Wait until April 1, then by all means "put them in a .txt file".

Could you elaborate on this? What is a reorg policy, and are there unpredictable variants?
I would have guessed that the decline in reorg frequency and depth is the result of lower miner inter-latency.

As is often the case, the problem as stated is massively underspecified.

For instance:
- Are your target addresses are correlated, perhaps even strictly sequential (in which case the solution is trivial)?
- How large is the dataset; is it feasible to transfer it to several kernels for parallellisation?
- Is the dataset mutable, or is it feasible to perform an initial, heavy transformation (in which case perfect hashing might outperform probabilistic/Bloom)?

Et cetera. A general database engine is almost certainly not optimal in either case.

Indeed it is. I got it from a fellow forum user in another thread. It looks live, but it hasn't been updated since 2020.

Right you are again, I lazily relied on english WP.

With gigabyte block sizes, I understand and agree completely.

Appearantly this topic was created accidently, and it's title isn't really matching the first posts. My interpretation of the OP has been that, as the total chain size (not individual block sizes) grows ever larger, prohibitive hard- and netware costs may cause a (further, potentially massive) decline of full nodes.

With current bitcoin block sizes, I still think hybrid pruning (as per my posts above) might be part of a solution to that potential problem.

In my case, because with a gigantic chain size, decentralisation is at risk (see above regarding costs etc).

This scares me. What you're describing is mitigation by centralisation, forced trust and forced custody. Unless you're proposing that every user should run her own L2 node, in which case we're back where we started (prohibitive costs).

Unfortunately (in my view), this separation between "users" and "miners" already is a fact. The original reason to use proof of work instead of simply counting IP addresses (to determine majorities) was to mitigate the fact that certain actors are able to allocate huge numbers of addresses. The scenario you're describing is magnitudes worse but not at all unlikely, I'm afraid. It's not really the topic of this thread though (but very related, I agree).

Sure, the terms are fuzzy. In this case, I contrasted forks with local chain reorgs. YMMV, but I know of only 6 global and sustained forks (excluding XT, BTH, Gold etc): 2010 (overflow bug), 2013 (db migration fix), 2015 (DER encoding), 2017 (Segwit), 2018 (double spending fix), 2021 (Taproot). Few and big enough that I would advocate revalidation, pruned node or not.

The "other kind" of reorgs, where blocks get orphaned because two or more blocks are mined almost at the same time, has as far as I can tell not occurred since 2019. It used to be much more frequent and might increase again (if f.x. inter-miner latency went up again), but with reasonable pruning parameters, this shouldn't affect a pruned node specifically, as far as I can tell; as soon as a new block is mined (from either chain), the tie is broken (in all likelyhood).

I take it you don't share the Satoshi Vision I think BSV has a max block size of 128 MB, and if we change BCT rules to allow gigabyte blocks, my current viewpoints may indeed be invalidated. I still fail, however, to grasp why a larger block size would cause orphans more frequently, let alone in chains of more than a hundred blocks at a time - how could a larger block size stop nodes from receiving any and all blocks from a competing chain in more than a thousand minutes? Especially since the competing chain actually worked harder for 625+ coins (if not, no split and no issue)?

Precisely! That a gigantic blockchain is no small task is the point of this thread   I'm suggesting that maybe pruned nodes, in combination with cheap, slow, non-transactional storage of pruned-away data, might help alleviating part of the consequences.

This sentence is not written by Molière (if it was, I'm sure that the Administrator, in his wisdom, would sense its significance and leave it alone).

Got you. For a full verification, this is likely the most efficient way, and also how my own implementation works. After that, new blocks are verified up towards the root until a previously verified branch is reached.

Instinctively, I would say that chain splits present no prune-specific vulnerabilities; even in the extreme case that the split is so far towards the root that it can't be reached from a pruned node's partial branches, there's no risk of false verification. In any case, chain splits are so rare and contentious that it would be prudent to revalidate from scratch after each one, pruned node or not.

Perhaps you mean local chain reorganisations, not actual forks? A node may indeed discover a new, longer chain, orphaning previous blocks, but I don't see how this is prune-specific (given reasonable pruning parameters).

I'm sure you don't really mean this  

Yes, I agree on both. My hopeful guess is that a large enough share of transaction needs could be fulfilled by cheap, pruned nodes (with all blocks from, say, the last 6 months), and that the rest of the data might be stored (locally, hence trusted, or non-locally if partner trust can be established) in a cheap, slow, non-transactional manner.

What's DT1 and DT2 pray tell?

In my vocabulary, that's the root not the tip (I mention this since nodes regularly use the tip, but seldom the root). Nomenclature aside, you are truistically right in that one would have to download the full tree in order to locally verify it. And that's important; don't trust, verify.

For one-time verification purposes, there's no technical need to hold the full tree locally at the same time; each branch can be verified separately.

Also, once the validity of the full tree is established at a local node, little is gained from full revalidation. This is the principle which makes pruning possible in the first place.

Of course, the full tree still needs to exist, preferably at as many locations as possible. But not necessarly in an expensive (i.e. fast, locally complete and randomly accessible) form.

Good overview, thanks.

Thanks for the tip link. Right now I'm mostly concerned with mined blocks (but no longer part of the longest chain).


I wish we could be absolutely certain of this, then we wouldn't ever require more than 1 confirmation, or even PoW at all 

For small amounts, 1 confirmation seems to be commonly accepted.

I realise that it's uncommon, thanks to your link and some more research I can confirm that the number of "extinct" blocks dropped from several per year to only 2 in 2019, and actually none since then.

Unfortunately I can't build my system based on hope alone. How do existing payment systems handle the situation when a transaction in a previously confirmed block actually disappears (perhaps because of a malicious actor trying to double spend)?

Thanks! For both the link and the sentiment.
Am I reading the link right, or have I smoked too much?
Not a single block rejected for four (4) years?

bump

Right, I'll update the OP to be clearer on that, thanks!

Sure, [img]https://ztt.se/btc:quote[/img] now yields the real-time XBT average (disregarding forum image proxy caches).

Fixed, posts with date Today now also get a price (the XBX average from https://www.coindesk.com).

I hope there aren't any rules against necroposting (I'm still confused over why that's reviled).

Isn't pruning a partial solution?