Quote from: cypherdoc on July 04, 2015, 08:02:47 PM

the increased frequency of SPV mining has occurred precisely b/c of the more consistently filled 1MB blocks and deviant defensive strategies being employed to navigate that congestion...otherwise, you'd have to be arguing that what once wasn't a problem with blocks <1MB have to now be occurring precisely b/c 1MB is a magic number at which blocks are deemed "too big". what is the chances of that?

Cypher, you're brilliant!

Evidence of an effective blocksize limit: no protocol-enforced limit required

In this post we show that, given a few simplifying assumptions^*, the network will automatically enforce an effective blocksize limit. This effective limit scales in an automatic fashion with improvements in technology, without requiring an explicit limit being enforced at the protocol level.

Background: We learned from last night's fork that miners are incentivized to mine "empty" blocks while they work to process the previously solved block. This increases their revenue, as shown here. However, this also means that the maximum possible value of the average blocksize is reduced in proportion to the frequency of these "empty blocks." For example, if 10% of the blocks were guaranteed to be empty, the maximum value of the average blocksize would presently be 900 kB, rather than 1 MB. We show that as the average size of the blocks increases, the percentage of empty blocks increases in direct proportion, thereby providing a counterbalancing force that serves to limit the blockchain's growth rate. We will refer to the "maximum value of the average blocksize" as the effective blocksize limit.

Let τ be the time it takes to process a typical block and let T be the average block time (10 min). [CLARIFICATION: τ includes all delays between the moment the miner has enough information to begin mining (an empty block) on the block header, to the moment he's processed the previous block, created a new non-empty block template, and has his hashing power working on that new non-empty block]. The fraction of time the miner is hashing on an empty block is clearly τ / T; the fraction of the time the miner is hashing on a non-empty block is 1 - τ / T = (T - τ) / T. We will assume that every miner applies the same policy of producing empty SPV blocks for time 0 τ.

Under these conditions, the expectation value of the blocksize is equal to the expectation value of the blocksize on the interval 0
   S_effective = ~0 [(τ / T)] + S' [(T - τ) / T]
   = S' [(T - τ) / T] (Eq. 1)

The time, τ, it takes to process a block is not constant, but rather is assumed to depend linearly^** on the size of the block. Approximating the size of the previous block as S', we get:

   τ = k S'

where k is the number of minutes if takes to process on average, 1 MB of transactional data. Substituting this into Eq. (1) yields the following equation:

   S_effective = S' (T - k S') / T
   = S' - (k/T) S'²

This is the equation for a concave down parabola, as shown:

To find the maximum of this curve, we take its partial derivative with respect to S' and set it to zero:

   d S_effective / d S' = 1 - 2 k S' / T = 0

Solving the above equation for S' gives

   S' = T / (2 k)

This (S') is the blocksize that maximizes the transactional capacity of the network. Substituting this result back into our equation for the effective blocksize limit gives:

   S_effective = T / (4 k)

Some numbers:

Assume it takes on average 15 seconds^*** to process a typical 1 MB block (k =0.25 min / MB). Since T = 10 min, this means the maximum average blocksize (network capacity) is limited to:

   S_effective = T / (4 k) = (10 min) / (4 x 0.25 min / MB)
   = 10 MB.

QED. We've shown that there exists a limit on the maximum value of the average blocksize, due to the time it takes to process and verify a block, irrespective of any protocol enforced limits.

^*There's a few other assumptions that I don't detail above, but it's Saturday and I'm enjoying the sunshine.
^**Here we're also assuming that the amount of ECDSA verify operations in a block is roughly proportional to the block's size.
^***This is a guess. We should estimate it by looking at the ratio of empty blocks to non-empty blocks produced by F2Pool.

**********UPDATE**********

Quote from: TheRealSteve on July 05, 2015, 12:23:37 AM

the last 27,027 blocks (basically since jan 1st 2015), f2pool-attributed blocks: 5241, of which coinbase-only: 139
For antpool, this is 4506 / 246.
See also: Empty blocks [bitcointalk.org]

Awesome! Thanks!!

We can estimate the average effective time it takes to process the blocks, then, as

   τ ~= T (N_empty / N_notempty)
   ~= T (N_empty / (N_total - N_empty))

F2Pool:

   ~= (10 min) x [139 / (5241 - 139)] = 16.3 seconds

AntPool:

   ~= (10 min) x [246 / (4506 - 246)] = 34.6 seconds