Quote from: solex on January 05, 2015, 11:14:15 PM

Quote from: cypherdoc on January 05, 2015, 05:11:09 PM

with anything new like this, i'm interested in knowing the upper and lower bounds of the IBLT data size.

lower bound: theoretically, as the UTXO set difference shrinks to zero with 0 network latency, the IBLT will shrink similarly but can never reach zero since it has to at minimum relay enough data to convey the exact subset of tx's included in the miner's block. how small can this data get esp in relation to the avg block size now?

upper bound: if the UTXO set difference is 100%, how big does the IBLT data size get? or does the entire IBLT concept fall apart at some intermediate set difference?

OK, so the bounds.

Note that UTXO is unspent tx outputs held in the blockchain, but unconfirmed tx are in node mempools, pending inclusion into the blockchain by miners.

man, that is depressing when one sees "un_" and extrapolates everything to mean the same thing after that Tongue

of course you're right.

but aren't UTXO also held in memory?:

UTXO are tracked by every full-node bitcoin client in a database held in memory, called the UTXO set or UTXO pool.
http://chimera.labs.oreilly.com/books/1234000001802/ch05.html#tx_inputs_outputs

Quote

What Satoshi achieved with his PoW blockchain breakthrough was consensus on the whole transaction history, and resulting UTXO. I think of that as the steel-work in a skyscraper. What IBLT does is go to further by enforcing consensus on unconfirmed transactions, it pours concrete into that steel reinforcing. This is a significant improvement to cryptocurrency, but it is truly effective only at high volumes, volumes too high for the existing standard method of block propagation.

i assume you mean high volumes of unconfirmed tx's. if so, how do we make the transition from the existing standard low volume method to IBLT?

Quote

The reason is that right now the whole network could agree on the same 2000 unconfirmed tx, real-world business, but the next block mined can contain none of them. It could be full of previously unknown gambling dice-bot spam tx, a set which is 100% different. Because these tx validly spend UTXO, then the block is accepted, and the 2000 unconfirmed tx have to wait for the next block.

how is this possible? i thought the unconf tx sets differences were supposed to be currently quite low which is the reason for IBLT in the first place? (sounds like you're saying we, in fact, don't have enough volume to make IBLT practical as of today)

Quote

However, when blocks are too big to propagate in the existing method, and require IBLT, then miners have to rely on the majority of the nodes knowing their unconfirmed tx in advance, they can only include a few unknown tx as the more unknowns they include the more likely their IBLT will fail to get decoded, and it gets rejected.

so how do miners know which unconf tx's are known vs unknown, ie, which to incl in the IBLT to enhance block acceptance ?

Quote

I expect IBLT blocks to be a constant size, starting at about 500KB or 1MB and staying at the same size for a long time, then increasing in steps, perhaps doubling after a number of years, just as the block reward halves periodically. A 1MB IBLT contains about 1500 differences, which means a 1% difference in mempools allows for 150,000 tx or 60MB disk blocks. A 32MB IBLT could support 3GB disk blocks, or 20,000 TPS, as by then differences of <1% should be the norm.

how do you arrive at 500KB or 1MB? or are you just using the current 1MB block limitation of today into which you would fit the IBLT? so are you saying that a 1MB sized IBLT equals 1500 diffs or an estimated 1% difference in unconf tx sets across network nodes or an equivalent 150,000 tx's block?