Search content
Sort by
Showing 6 of 6 results by tdryja
Re: Increasing the block size is a good idea; 50%/year is probably too aggressive
Did you read my "blocksize economics" blog post?
I don't understand why you think MAX_BLOCK_SIZE necessarily has anything to do with "supporting mining" (aka securing the network).
I don't understand why you think MAX_BLOCK_SIZE necessarily has anything to do with "supporting mining" (aka securing the network).
I can't speak for NewLiberty, but I have certainly read it, and agree with the majority of what you've written. The part about "Block Subsidy, Fees, and Blockchain Security" is most relevant here. I agree that as it stands, there is no guarantee that 1MB blocks would be full of high value transactions, and no guarantee that 1GB blocks would be full enough of low value transactions to secure the network.
However, if the max block size is linked to the transaction fees, we can at least know that the 1GB block does have sufficient fees, because the size would contract if it didn't. The other scenario -- a half empty 1MB block with minimal fees on a few large transactions -- implies that Bitcoin has either failed or been superseded, at which point the max block size is not relevant.
What stops this from happening:
Big miners accept off-blockchain payments from big merchants and exchanges that want their transactions confirmed. They are included in very small blocks with zero fees. The blocksize stays at 1MB forever.
Big miners accept off-blockchain payments from big merchants and exchanges that want their transactions confirmed. They are included in very small blocks with zero fees. The blocksize stays at 1MB forever.
2 things: 1 which stops it from happening, and 1 which means it could happen anyway.
This scenario supposes that 1MB is sufficient to maintain the miner / merchant cartel's transactions, which may not be the case, but is plausible. What is implausible is that every member of this cartel of miners continues to reject a vast mempool of outsider fee paying transactions. Thousands of merchants saying "shut up and take my bitcoins! include my tx!" and the miners all say "No!", maintain their cartel, and deny themselves that money? Or, if they try to on-board these merchants into their cartel, the 1MB block isn't big enough anymore. Similarly for merchants, are they getting a better deal with the cartel? If so, great, but why is the cartel being nice to the merchants; it's much more likely that the merchants would hate the cartel and try to get their transactions in a cheaper, independent block.
Why maintain membership in the cartel if you make less money? One of those two groups (miners, merchants) must be making less money.
This type of cartel is also possible with an open-loop exponential expansion of max block size. The majority of the miners can stick to 1MB blocks, and reject blocks with transactions not in their cartel. >50% of miners need to participate in this cartel to effectively push down the median fees. It doesn't make rational sense (unless megabytes are extremely expensive) in this case either, but if we worry about a malicious majority mining cartel, is still doable.
I think an open-loop larger block size would probably be fine, but it involves a lot of extrapolation. Maybe computers get way better really fast, and 1GB is laughably small. Or maybe they stay the same, and 1GB is too large, meaning the networking and storage costs of mining exceed the sha256 costs, centralizing mining. I think a closed-loop feedback system based on median aggregate transaction fees is able to reduce these risks.
Re: Increasing the block size is a good idea; 50%/year is probably too aggressive
trout:
empty blocks are possible now, and not a big deal. They become very expensive longer term as fees take over the block reward; an empty block could have no or negligible reward. If the median is used, this attack will have minimal effect on the network, while costing the attacker 1 BTC per empty block. I dont think we need to worry about an attack which is very expensive to for the attacker, and has no appreciable effect on the network.
I agree that it may be easier to form a majority cartel if the only thing at stake is block size. But a majority cartel of miners can pretty much do this anyway; they just tell everyone Hey guys, the new max block size is 1GB. Were all moving our mining power there, youd best update your clients.
Basically I think worrying about a majority of miners doing something you dont want them to is beyond the scope of the problem. And if they all want to have huge blocks and put all my transactions in there for free, I for one welcome our new benevolent mining overlords
David Rabahy:
The idea of only allowing known transactions into a block has been discussed before, but has been found unworkable. The purpose of the block is to achieve consensus on which transactions have happened. Presupposing consensus on the set of transactions removes the need for the block. In other words, if all the miners already agree on whats going to be in the next block, why bother broadcasting it to each other?
There are different ways to try to make that work, and Ive discussed it with several people, but I think its fundamentally incompatible with Bitcoins current consensus system.
empty blocks are possible now, and not a big deal. They become very expensive longer term as fees take over the block reward; an empty block could have no or negligible reward. If the median is used, this attack will have minimal effect on the network, while costing the attacker 1 BTC per empty block. I dont think we need to worry about an attack which is very expensive to for the attacker, and has no appreciable effect on the network.
I agree that it may be easier to form a majority cartel if the only thing at stake is block size. But a majority cartel of miners can pretty much do this anyway; they just tell everyone Hey guys, the new max block size is 1GB. Were all moving our mining power there, youd best update your clients.
Basically I think worrying about a majority of miners doing something you dont want them to is beyond the scope of the problem. And if they all want to have huge blocks and put all my transactions in there for free, I for one welcome our new benevolent mining overlords
David Rabahy:
The idea of only allowing known transactions into a block has been discussed before, but has been found unworkable. The purpose of the block is to achieve consensus on which transactions have happened. Presupposing consensus on the set of transactions removes the need for the block. In other words, if all the miners already agree on whats going to be in the next block, why bother broadcasting it to each other?
There are different ways to try to make that work, and Ive discussed it with several people, but I think its fundamentally incompatible with Bitcoins current consensus system.
Re: Increasing the block size is a good idea; 50%/year is probably too aggressive
David Rabahy:
I generally try not to think in dollar terms about the economic issues in Bitcoin. If there is a feedback system such that block rewards from tx fees tends towards 1BTC / block, the blocks could potentially be quite large; 100MB/block, or with your estimates 179,000 transaction, at a cost of 5.5 uBTC per tx. More transactions trying to fit into a 100MB block will tend to push up the per tx fee, which would to expand the max block size to say 110MB, which pushes the fees per tx back down such that the new 110MB blocks are just about full of txs at a 5.4 uBTC / tx fee, still earning 1BTC per block.
trout:
I address this in my initial post and go into detail below.
2112:
I've thought about the same set of changes, and have decided that it's probably too much of a change to practically push through into Bitcoin. Something where the miner of block n gets 1/2 the tx fees, and the miner of block n+10 gets the other half would both incent inclusion of high-fee transactions, as well as eliminate the risk that miners would pay fees to themselves. Such a fundamental change however is probably impractical, as it would be dismissed by many as "not Bitcoin". Integrating something like p2pool is also quite complex and will be viewed as too risky.
NewLiberty:
I wasn't clear enough about this in my post, but I meant that the new epoch's block size to be a function of the previous one, just like the difficulty adjustments. Difficulty adjustments don't actually care about hash rate, just the time taken for the 2016 block epoch, and a relative difficulty adjustment is made based on the divergence from the two week target. Similarly, I agree that max block size should use transaction fees as a relative adjustment factor. I mention bounds of this adjustment below.
-
Simply using median transaction fees per block over the past epoch is hopefully simple and straightforward enough to be accepted by people, and does not have significant incentive problems.
There are two ways this can be 'gamed' by miners. The way that is most dangerous to the non-mining users of the network would be for miners to artificially limit block sized to a small value, in the hopes that they would profit from high transaction fees. Doing this requires malicious collusion among miners (in excess of that in a proof-of-idle system which I've written about) and in a situation where most of the miners are trying to harm bitcoin, we're already in much bigger trouble. In practice miners will grab all the fees they can fit into a block, especially if they know the next miner will do the same.
The more problematic way a malicious miner can 'game' this is by paying fees to itself. Using thresholds, or the median instead of mean, or some other mathematical way to cut out the outliers may be helpful. I like using the median block reward -- it's really quite simple and would prevent anyone with <50% of the hash power from accomplishing much. And the assumption in all of this is that there is no >50% miner.
If I miner did somehow push the fees and blocksize up, that miner could then publish large blocks in an attempt to spam / DoS the network. That's the only real threat, and it could be very costly and slow for the miner to accomplish. Unlike the difficulty adjustment, which is bounded at 0.25X to 4X, the max block size adjustment could have a much tighter bound, like 10%, so that it would take months to double the max block size.
I think this is simple and straightforward enough that miners, developers, and bitcoin users can read it, understand it, and be OK with it. I also think that it's safe long-term, and doesn't require human intervention once set up, regardless of how computer technology evolves (or fails to) over the next few decades.
Thanks to everyone who's read and commented on this; I actually thought of this a few years ago and mentioned it to people but never had gotten any attention. My personal opinion is that Gavin's idea of just increase blocks based on a guess of continuance of Moore's law would probably work fine... but I like my idea a little better Thanks for the comments.
I generally try not to think in dollar terms about the economic issues in Bitcoin. If there is a feedback system such that block rewards from tx fees tends towards 1BTC / block, the blocks could potentially be quite large; 100MB/block, or with your estimates 179,000 transaction, at a cost of 5.5 uBTC per tx. More transactions trying to fit into a 100MB block will tend to push up the per tx fee, which would to expand the max block size to say 110MB, which pushes the fees per tx back down such that the new 110MB blocks are just about full of txs at a 5.4 uBTC / tx fee, still earning 1BTC per block.
trout:
I address this in my initial post and go into detail below.
2112:
I've thought about the same set of changes, and have decided that it's probably too much of a change to practically push through into Bitcoin. Something where the miner of block n gets 1/2 the tx fees, and the miner of block n+10 gets the other half would both incent inclusion of high-fee transactions, as well as eliminate the risk that miners would pay fees to themselves. Such a fundamental change however is probably impractical, as it would be dismissed by many as "not Bitcoin". Integrating something like p2pool is also quite complex and will be viewed as too risky.
NewLiberty:
I wasn't clear enough about this in my post, but I meant that the new epoch's block size to be a function of the previous one, just like the difficulty adjustments. Difficulty adjustments don't actually care about hash rate, just the time taken for the 2016 block epoch, and a relative difficulty adjustment is made based on the divergence from the two week target. Similarly, I agree that max block size should use transaction fees as a relative adjustment factor. I mention bounds of this adjustment below.
-
Simply using median transaction fees per block over the past epoch is hopefully simple and straightforward enough to be accepted by people, and does not have significant incentive problems.
There are two ways this can be 'gamed' by miners. The way that is most dangerous to the non-mining users of the network would be for miners to artificially limit block sized to a small value, in the hopes that they would profit from high transaction fees. Doing this requires malicious collusion among miners (in excess of that in a proof-of-idle system which I've written about) and in a situation where most of the miners are trying to harm bitcoin, we're already in much bigger trouble. In practice miners will grab all the fees they can fit into a block, especially if they know the next miner will do the same.
The more problematic way a malicious miner can 'game' this is by paying fees to itself. Using thresholds, or the median instead of mean, or some other mathematical way to cut out the outliers may be helpful. I like using the median block reward -- it's really quite simple and would prevent anyone with <50% of the hash power from accomplishing much. And the assumption in all of this is that there is no >50% miner.
If I miner did somehow push the fees and blocksize up, that miner could then publish large blocks in an attempt to spam / DoS the network. That's the only real threat, and it could be very costly and slow for the miner to accomplish. Unlike the difficulty adjustment, which is bounded at 0.25X to 4X, the max block size adjustment could have a much tighter bound, like 10%, so that it would take months to double the max block size.
I think this is simple and straightforward enough that miners, developers, and bitcoin users can read it, understand it, and be OK with it. I also think that it's safe long-term, and doesn't require human intervention once set up, regardless of how computer technology evolves (or fails to) over the next few decades.
Thanks to everyone who's read and commented on this; I actually thought of this a few years ago and mentioned it to people but never had gotten any attention. My personal opinion is that Gavin's idea of just increase blocks based on a guess of continuance of Moore's law would probably work fine... but I like my idea a little better
Thanks for the comments.Re: Increasing the block size is a good idea; 50%/year is probably too aggressive
My 2uBTC on this issue:
Instead of guessing the costs of the network via extrapolation, code in a constant-cost negative feedback mechanism. For example, similar to difficulty adjustments, if mean non-coinbase block reward > 1 BTC, increase max size. If mean block reward < 1 BTC, decrease max size (floor of 1MB).
Here's why I think this is a long term solution. With Bitcoin, "costs" and "value" have a very interesting relationship; currently with mining, the costs to run the network are determined by the exchange value of a bitcoin. Long term, the block size constrains both the cost and value of the network. By "long term", I mean 100 years from now. Long term, there's no more coinbase reward. So miners compete for transaction fees. Limited block size causes transactors to compete for space in the block, driving up the fees. An unlimited block size would, without other costs, tend to drive fees to near-zero, and then there's not enough incentive for miners to bother, and the security of the system is compromised. That's the death spiral idea anyway, which may not actually happen, but it's a legitimate risk, and should be avoided. The value and utility of bitcoin today has a lot to do with the probability that it will have value in 100 years.
Max block sizes doubling every two years makes them pretty much unlimited. Capping after 20 years is also a big guess. That also extrapolates Moore's law for potentially longer than the law keeps going. Gigabit ethernet is what, 15 years old? And that's what every PC has now, I've never seen 10G over copper ethernet. Reliance on everything else becoming awesome is a very fragile strategy.
An issue I have with expoentially increasing block size, or static block size, is there's no feedback, and can't respond to changes in the system. The block size in many ways determines the value of the network. All else being equal, a network that can handle more transactions per day is more useful and more valuable.
I think that similar to the current system of mining costs determined by bitcoin value, block propagation, verification and storage should be determined by how much people are willing to pay. If transaction fees are high, block space is scarce, and will expand. If transaction fees are low, block space is too cheap, and the max block size will expand.
This fixes a cost independent of the mining coinbase reward, allowing for sustainable, predictable mining revenue. The issue is we would have to come up with a number. What should it cost to run the bitcoin network? 1% of M0 per year? That would be 210,000 coins per year in transaction fees to miners. That would be about 3BTC per block.
0.5% M0 annually would be 1.5BTC per block, and so on. This would be a ceiling cost; it could cost less, if people didn't make too many transactions, or most things happened off-blockchain, and the blocks tended back towards the 1MB floor. It would effectively put a ceiling on the maintenance cost of the network, however; if blocks were receiving 6BTC in fees, the size would double at the next difficulty adjustment, which would tend to push total fees down.
If you wanted to get fancy you could have hysteresis and non-linearity and stuff like that but if it were up to me I'd keep it really simple and say that max block size is a linear function of the previous epoch block rewards.
This can be "gamed" in 2 ways. It can be gamed to a limited extent by miners who want to push up the max block size. They can pay a bunch of fees to themselves and push up that average. I can't think of a clean way to get rid of that, but hopefully that's OK; isn't it the miners who want smaller blocks anyway? If miners are competing for larger blocks, why would the non-mining users complain? The only issue is one miner who wants larger blocks, and everyone else wants smaller ones. Maybe use median instead of mean to chop out malicious miners or fat-finger giant transaction fees.
It can also be gamed the other way. Your transaction fee is 0, but you have some off-channel account with my mining group which includes all your txs for a flat monthly rate. This also seems unlikely; if it were more expensive that way, transactors would stop using the off-channel method and just go to the open market for transaction inclusion. If it were cheaper, why would the miner forgo that revenue?
So if I ran this whole Bitcoin thing (which would defeat the point..., that's what I would do. The question is how much it should cost. 1BTC per block sounds OK, it's nice round number. That's 50K BTC per year for the miners.
I'd welcome comments / criticism of why having such a feedback mechanism is a good or bad idea.
Instead of guessing the costs of the network via extrapolation, code in a constant-cost negative feedback mechanism. For example, similar to difficulty adjustments, if mean non-coinbase block reward > 1 BTC, increase max size. If mean block reward < 1 BTC, decrease max size (floor of 1MB).
Here's why I think this is a long term solution. With Bitcoin, "costs" and "value" have a very interesting relationship; currently with mining, the costs to run the network are determined by the exchange value of a bitcoin. Long term, the block size constrains both the cost and value of the network. By "long term", I mean 100 years from now. Long term, there's no more coinbase reward. So miners compete for transaction fees. Limited block size causes transactors to compete for space in the block, driving up the fees. An unlimited block size would, without other costs, tend to drive fees to near-zero, and then there's not enough incentive for miners to bother, and the security of the system is compromised. That's the death spiral idea anyway, which may not actually happen, but it's a legitimate risk, and should be avoided. The value and utility of bitcoin today has a lot to do with the probability that it will have value in 100 years.
Max block sizes doubling every two years makes them pretty much unlimited. Capping after 20 years is also a big guess. That also extrapolates Moore's law for potentially longer than the law keeps going. Gigabit ethernet is what, 15 years old? And that's what every PC has now, I've never seen 10G over copper ethernet. Reliance on everything else becoming awesome is a very fragile strategy.
An issue I have with expoentially increasing block size, or static block size, is there's no feedback, and can't respond to changes in the system. The block size in many ways determines the value of the network. All else being equal, a network that can handle more transactions per day is more useful and more valuable.
I think that similar to the current system of mining costs determined by bitcoin value, block propagation, verification and storage should be determined by how much people are willing to pay. If transaction fees are high, block space is scarce, and will expand. If transaction fees are low, block space is too cheap, and the max block size will expand.
This fixes a cost independent of the mining coinbase reward, allowing for sustainable, predictable mining revenue. The issue is we would have to come up with a number. What should it cost to run the bitcoin network? 1% of M0 per year? That would be 210,000 coins per year in transaction fees to miners. That would be about 3BTC per block.
0.5% M0 annually would be 1.5BTC per block, and so on. This would be a ceiling cost; it could cost less, if people didn't make too many transactions, or most things happened off-blockchain, and the blocks tended back towards the 1MB floor. It would effectively put a ceiling on the maintenance cost of the network, however; if blocks were receiving 6BTC in fees, the size would double at the next difficulty adjustment, which would tend to push total fees down.
If you wanted to get fancy you could have hysteresis and non-linearity and stuff like that but if it were up to me I'd keep it really simple and say that max block size is a linear function of the previous epoch block rewards.
This can be "gamed" in 2 ways. It can be gamed to a limited extent by miners who want to push up the max block size. They can pay a bunch of fees to themselves and push up that average. I can't think of a clean way to get rid of that, but hopefully that's OK; isn't it the miners who want smaller blocks anyway? If miners are competing for larger blocks, why would the non-mining users complain? The only issue is one miner who wants larger blocks, and everyone else wants smaller ones. Maybe use median instead of mean to chop out malicious miners or fat-finger giant transaction fees.
It can also be gamed the other way. Your transaction fee is 0, but you have some off-channel account with my mining group which includes all your txs for a flat monthly rate. This also seems unlikely; if it were more expensive that way, transactors would stop using the off-channel method and just go to the open market for transaction inclusion. If it were cheaper, why would the miner forgo that revenue?
So if I ran this whole Bitcoin thing (which would defeat the point...
, that's what I would do. The question is how much it should cost. 1BTC per block sounds OK, it's nice round number. That's 50K BTC per year for the miners.I'd welcome comments / criticism of why having such a feedback mechanism is a good or bad idea.
Hi - I'm interested in developing a plugin for Electrum.
Is there any documentation on where to start with this? I've cloned she github repo on my local machine, and can compile and run everything. There's a plugins folder, and I've looked through there. Can I just put any old .py file in there? I haven't found any guide / HOWTO / readme or anything about the plugins, is there anything and I've missed it?
So far I've tried to duplicate the "virtual keyboard" plugin, to make a "bad keyboard" which only has the letter A. Copying and modifying the virtualkeyboard.py file works, and I get a checkbox in the plugins menu in electrum. It seems to work and I can modify it, but is there any defined API? Do I have to modify other files in the lib folder and so on, or should I be able to do everything I need to from the plugins folder?
I'm looking into making some client side multi-sig stuff and thought the idea of Electrum plugins seemed promising. If there are other bitcoin clients anyone things might be more suitable or modifiable I'm also open to those suggestions before I get to deep into the Electrum codebase (which is a great project, but I'm hoping with a plugin I can leave that side unmodified)
Thanks for any info / links / help
Is there any documentation on where to start with this? I've cloned she github repo on my local machine, and can compile and run everything. There's a plugins folder, and I've looked through there. Can I just put any old .py file in there? I haven't found any guide / HOWTO / readme or anything about the plugins, is there anything and I've missed it?
So far I've tried to duplicate the "virtual keyboard" plugin, to make a "bad keyboard" which only has the letter A. Copying and modifying the virtualkeyboard.py file works, and I get a checkbox in the plugins menu in electrum. It seems to work and I can modify it, but is there any defined API? Do I have to modify other files in the lib folder and so on, or should I be able to do everything I need to from the plugins folder?
I'm looking into making some client side multi-sig stuff and thought the idea of Electrum plugins seemed promising. If there are other bitcoin clients anyone things might be more suitable or modifiable I'm also open to those suggestions before I get to deep into the Electrum codebase (which is a great project, but I'm hoping with a plugin I can leave that side unmodified)
Thanks for any info / links / help
Re: Explanation needed on Proof of Idle - Optimal Mining Strategy
Hi - I'm the guy who gave the talk. Just made an account here to reply.
Franky1- I seem to have touched a nerve. I'm hesitant to address your posts at all (referring to this idea as "a basement dwelling freeloader's wet dream" makes it clear that this isn't any kind of rational debate) but will optimistically respond to the post to the extent possible.
I don't understand your "bad actor" scenario. There is no reason for a miner to "contract with themselves", because the contract transactions are not made public, and don't appear on the blockchain until one of the two nLockTimes becomes valid. Why would you send coins to yourself? Nobody sees the proof of power responses except the node creating the bounty.
Also, if smaller miners who were hoping to idle their hash power and receive a bounty see that the network is running at a high rate, they will assume the bounties will not be fulfilled and adjust their hash rates to the optimum without a bounty payout. So if at 00:03 am, some mining pool starts running full blast, well, everyone can see that, and the whole PoI contract is off, mining continues as before, and the bounties are returned to the sender a few days later. The argument of this talk is that this is not an optimal choice for a large miner to make. Granted, many miners are not rational and can try to do dumb things. This doesn't address that, and assumes rational actors trying to maximize their bitcoin income.
Sorry if this wasn't clear in the talk, I'll try to have a nice PDF of it out soon. The slides linked on the youtube page explain most of it though.
This does not help with decentralization at all in fact. I'm not advocating this idea, I'm just saying that my research has indicated that it is possible, optimal, and likely to be inevitable if the right conditions are met (mainly a low opex profitability and somewhat stable network with large mining entities). Miners would still need to have equipment to pose credible threats to mine, but would not have to mine at full capacity.
(Also this should possibly be moved to Development & Technical Discussion, but I didn't create the thread.)
I'll post responses tomorrow if people have other questions. Thanks for the interest!
Franky1- I seem to have touched a nerve. I'm hesitant to address your posts at all (referring to this idea as "a basement dwelling freeloader's wet dream" makes it clear that this isn't any kind of rational debate) but will optimistically respond to the post to the extent possible.
I don't understand your "bad actor" scenario. There is no reason for a miner to "contract with themselves", because the contract transactions are not made public, and don't appear on the blockchain until one of the two nLockTimes becomes valid. Why would you send coins to yourself? Nobody sees the proof of power responses except the node creating the bounty.
Also, if smaller miners who were hoping to idle their hash power and receive a bounty see that the network is running at a high rate, they will assume the bounties will not be fulfilled and adjust their hash rates to the optimum without a bounty payout. So if at 00:03 am, some mining pool starts running full blast, well, everyone can see that, and the whole PoI contract is off, mining continues as before, and the bounties are returned to the sender a few days later. The argument of this talk is that this is not an optimal choice for a large miner to make. Granted, many miners are not rational and can try to do dumb things. This doesn't address that, and assumes rational actors trying to maximize their bitcoin income.
Sorry if this wasn't clear in the talk, I'll try to have a nice PDF of it out soon. The slides linked on the youtube page explain most of it though.
This does not help with decentralization at all in fact. I'm not advocating this idea, I'm just saying that my research has indicated that it is possible, optimal, and likely to be inevitable if the right conditions are met (mainly a low opex profitability and somewhat stable network with large mining entities). Miners would still need to have equipment to pose credible threats to mine, but would not have to mine at full capacity.
(Also this should possibly be moved to Development & Technical Discussion, but I didn't create the thread.)
I'll post responses tomorrow if people have other questions. Thanks for the interest!