Quote from: DeathAndTaxes on November 20, 2013, 05:47:53 PM

Quote from: Gavin Andresen on November 20, 2013, 06:42:38 AM

DeathAndTaxes: I'm normally impressed with your posts, but I think you've got some details wrong.

Thanks and I am happy to be corrected.

Quote

First, RE: the orphan cost of transactions: Decker/Wattenhofer measured 80ms for a 1K bigger block.
...
So 3.3 millies per kilobyte is the orphan cost.

I didn't know about this source. I recall (but can't seem to find a cite) with a much lower estimated cost although it may have simply been an error on my part ~0.4 mBTC vs ~4 mBTC.

Quote

It should be fairly easy to get about another factor of about 10-20 reduction in orphan costs.

Correct me if I am wrong but a rather simple block broadcast improvement would be to not include the tx just include their hashes. Since a well connected miner aware of a given tx X should already have relayed that tx X to his peers there is little need to include the full tx in the block message. Instead block message could just consist of the header and a list of tx hashes. If average tx size is 400 bytes and the SHA-256 hash is 32 bytes that alone could cut the orphan cost of a tx by 90% to ~0.3 mBTC. In reality since the tx hash is just being compared to the UXTO a more involved modification would be for the block message to include a truncated hash of the tx. This wouldn't represent a security risk as the actual merkle tree would involve the full SHA-256 hash. Using say the first 64 bits of the SHA-256 hash would still make collisions essentially highly unlikely events and would reduce the orphan cost by another 4x to <0.1 mBTC.

Quote

And as I said in another thread, if EVERYBODY produces larger blocks then EVERYBODY bears the increased orphan cost, and the result is better for everybody . There is a fixed number of new bitcoins to be earned, regardless of the orphan rate; everybody's share of that fixed number will be the same if everybody has a slightly higher orphan block rate. But everybody will earn more fees, and their bitcoins will be worth more because bitcoins will be more useful.

Agreed. Hopefully pools and major solo operators can see that their long term profitability is based MORE on the growth of Bitcoin than the short term orphan cost. Hopefully large pools are aware they don't need universal support. If 70% of hashpower agrees to produce larger blocks (say 500KB avg) for the good of the network. It cuts the orphan cost by 70%. Still I think the orphan cost does highlight the reality that miners are going to be increasingly reluctant to devote more space to free tx. This is something users will have to come to grips with. Including a fee and waiting hours or days is unacceptable (although it is often for non-fee reasons) however no fee tx should be considered charity by users and if someone does you an act of charity in an hour, or day, or week well you got what you paid for. The default behavior of most clients should probably be changed to include the "min fee" on ALL txs not just low priority ones. If users want to they could change this but they should be warned "Including no tx fee may result in delayed confirmation times". Enforcement for relaying at node level should still only be on low priority.

It is somewhat ironic that this is more of an issue due to the higher exchange rate. The min fee on low priority tx was lowered due to rising exchange rate. Today 3.3 mBTC is ~$1.50. Ouch. However if Bitcoins were worth less it would be less of a cost. Since Bitcoin is often used as a proxy for USD a 5 mBTC fee (which more than covers orphan costs) is more viable at a lower exchange rate.

Couldn't it be possible for blocks to just include hashes of a file, without including all the inputs, outputs, etc, just minimal info, the nuonce (that will be longer than the it is now), timestamp, maybe number of inputs/output and bitcoins transfered.

So bitcoin will work with a DHT just like bittorrent,

the miner will broadcast just this mini-blocks that matches the difficulty, then the rest of the nodes will accept it as valid, and start working in the next block, and simultaneously, ASK for the "file with all the transactions in it". (this file is just like the block is now.)

So this fix is just like the "announce fix" but with proof of work.

This small fix, together with an option to "just download the last X blocks", and the proper implementation of said DHT, allowing to lookup by block and also query the network for any input, just like the client actually does with the database, could make bitcoin infinitely scalable without the block propagation issue.

Or a quick&dirty fix could be just make all blocks the same 1 mb size and fill a field with info from the last block(s) in order for them to meet said size, then the block size propagation time will be a non issue, and this additional data can be discarded as it is just a filler.