Thanks - my bad. I confused your point about the challenge (or benefits) of integrating p2pool into the reference client ("…these cheap machines would still be out of the question… The main client really struggles on the 512Mb RAM raspberry pi") with your separate/main point regarding variance. Thanks for being patient with me as I become more familiar with the details of p2pool and the mining landscape.

It sounds like there are some good discussions happening among people who know a lot more about this than I, so I'm happy about that. I'm just hoping that an effective solution can be engineered, deployed, and adopted before it's too late

I understand what you're saying. If we're just talking about variance, then yes, I agree variance is a problem, especially if it forces a large percentage of miners to discontinue mining. But when it comes to controller hardware/software considerations, I don't think the variance argument really applies, because if tomorrow the protocol/software necessitated a slight upgrade in controller hardware in order to keep current mining hardware functional, I believe most would upgrade very quickly (if the choice is between turning your mining investment into a paperweight and spending a few bucks to upgrade the controller hardware, it's not a tough decision). The cost of controller hardware is and likely always will be a tiny fraction of the total cost of a competitive mining rig. For that reason, I still believe that the solution should not necessarily cater to any specific hardware configuration.

To put it another way, let's fast forward 10 years to a time when (if Bitcoin is still around) all of the mining hardware you're referring to is completely useless for any kind of profitable mining. The state-of-the-art mining rigs dwarf all of today's machines in comparison. Are you going to be fighting the same battle then, arguing that since some miners from years ago relied on obsolete microcontrollers for controlling their mining hardware, development should still cater to those as a least common denominator?

Because, as you put it earlier, "more overall hashrate is better." Apparently that assertion trumps all other concerns.

How could the network possibly deal with those people shutting down obsolete mining rigs? /sarcasm

(I honestly don't mean any personal disrespect in this conversation - I'm just finding it difficult to understand the rationale for catering to any specific hardware configuration, especially a hardware configuration that seems so… misguided and cheap).

Yes, I understand the difference between a controller (which admittedly can run on very low-end hardware) and hashing power. My point is, if you were using a $35 piece of hardware to control a nuclear reactor (yes, I'm exaggerating a bit to make a point), and the reactor - or rather the entire energy grid - were at risk of meltdown without a very minor, known-working hardware upgrade (compared to the reactor) for the controller to support the software fix, would you be wasting time trying to figure out how to somehow re-architect the software to run on the old hardware just to save a couple of bucks?

Do you really think that forcing/encouraging that minimal upgrade would disrupt the current (non-existent) "balance of power" on the network?

No, I don't think that's comparing apples to oranges. If anything, I think the current mining situation is even more ridiculous given the fact that (assuming the network survives), all of the current hashing power mining behind those cheap controllers is going to become worthless in a year (or less) anyway.

Go ahead and spin your wheels trying to save a small subset of current mining hardware while you watch the network crash as all those ignorant miners  who purchased pi-controlled ASICs point their hashing power at "benevolent" centralized pools who would NEVER do anything harmful (because the miners don't know better and/or they can't run something better on their controllers). Don't complain that we didn't have enough time/resources to fix the problem. Talk about missing the forest for the trees...

I guess all I can do is echo gmaxwell's sentiment:



In your mind, mining can only either be controlled by a $2 microprocessor, or "the most dense and voluminous concentration of hashing power in a unit... requiring 240 volt electricity supplies, and thousands of watts of heat exhaustion"? Really, nothing in between?

I find it difficult to comprehend how someone could make a case for delaying or hindering adoption of a critical, network-saving change just for the sake of ridiculous hardware constraints like that mentioned above... unless there is a conflict of interest involved or some kind of financial incentive to profit from the deployment of such devices.

I... simply don't have the words to describe my dismay at this kind of flawed reasoning. If the only thing standing between Bitcoin's success and failure are a bunch of overpriced machines controlled by Raspberry Pis (engineered, by the way, to be underpowered and save *pennies* for the sake of bringing technology education to the less fortunate), then I'm sorry, Bitcoin is dead. End of story. We're talking about securing a global decentralized cryptocurrency, not learning to type "print('Hello, World!')".

Sorry for potentially misunderstanding the problem, but why is hardware - in any way - dictating the direction of software development here? Bitcoin itself is built to scale with changing hardware capabilities by automatically adjusting difficulty as needed. What makes a decentralized mining pool so different conceptually?

Why do you worry about forcing out a subset of current miners who will be forced out with time (and increasing difficulty) anyway? If all miners were forced onto p2pool tomorrow and people with less than (as an example) 1TH/s rigs are no longer competitive, would the remainder of the hashing power distribution really be any less diverse in terms of control than the current situation with GHash.IO? It seems to me that with a bit of time, everyone currently mining is going to be "forced out" anyway due to the competitive forces of hardware advancement. If a software change temporarily accelerates that process in an unbiased way for everyone on the network, how is that bad? The only way I can see it mattering significantly is if the % of miners forced out is very high (e.g., a majority of all miners) causing a slightly "unfair" advantage for those with access to the best hardware (since the marginal cost for additional ASIC hardware could buy a slightly bigger piece of the pie than it could previously), but that was true when the first ASIC rigs became available anyway. The market (in my opinion) has and will adjust if there is any such economic advantage for certain classes of mining hardware.

It sounds like there may be some interest in bringing p2pool (or a similar concept) closer to the reference Bitcoin implementation (whether that be a change in the protocol, bundling p2pool with the reference client, or simply giving p2pool a stronger online presence in connection with bitcoin.org).

I'd like to try and gauge interest in the various approaches to solving this problem, and perhaps (if it hasn't been done already) formalize some kind of plan of action.

The possible directions I'm seeing are (and these are not mutually exclusive):

• More discussion leading to a formal BIP submission with changes to the Bitcoin protocol and reference client. Then, I guess we wait and hope that either the core team can get to accepting and implementing that BIP sooner rather than later (understanding that there are numerous accepted BIPs whose priority seem to be uncertain), or someone else can step up to the challenge... makes we wonder though how many developers outside the core team there really are that have the expertise, familiarity with Bitcoin, and incentive to contribute such a fundamental change in the timeline needed.
• More discussion about improvements to p2pool (as the separate piece of software it is now) attempting to address any technical shortcomings of p2pool (scaling, speed, hardware requirements, etc).
• More discussion about improving the non-technical issues of p2pool - public awareness, user experience improvements, etc.
• Discussing ways to add resources/velocity to the speedy development of one or more of the solutions above (in the form of crowdfunding, offering bounties, raising developer awareness, etc).

So my question is: how can we best contribute to this issue being solved effectively in the quickest timeframe possible?

What do the core developers feel is most needed at this point? Can we all try and reach some kind of consensus as to how this should be addressed? I feel like unless we can (most of us anyway) concentrate our efforts on one of these things, we are wasting time and resources by chasing a number of different proposals. In theory (with open-source software like Bitcoin), a number of separate contributing groups can work on different solutions to the same problem, and I suppose the best of many potential solutions would naturally emerge, but in the case of Bitcoin, I get the feeling that the number of developers in a position to make these kinds of changes is somewhat limited, and we don't have the time to wait for a solution to just roll around "naturally".

Any thoughts or opinions on how best to move forward to address this concern?

I'm afraid that if we don't act quickly, these discussions will merely become artifacts of a failed cryptocurrency replaced by something better - all because we couldn't fix these kinds of issues fast enough.

I really appreciate all of the responses here.

So to summarize what I'm learning: p2pool is a fairly sound system in concept, but its weaknesses are that (1) it doesn't currently have the visibility or level of appeal to gain the attention it might deserve, and (2) even if it gained more attention and adoption, it probably wouldn't be able to compete with well-run centralized pools.

Does that sound about right?

So, if Bitcoin's current proof-of-work reward system encourages centralized pools to emerge and flourish (which are a threat to the system), is there really nothing better we can do than wait and watch the system fall apart in time? Surely there has to be a better solution...

If p2pool faces (from what I'm hearing) an uphill battle to gain any substantial adoption in its current form, then what about the original question I asked:

Would it be technically feasible to incorporate the concept behind P2Pool into Bitcoin itself [a forced change by the protocol]?


Carlton mentioned:

To me, seeing some miners forced out as a consequence (due to difficulty or hardware limitations) seems like a price worth paying for the network's survival.

Maybe I'm missing something obvious, but wouldn't such a change ultimately solve both of the issues (adoption and being competitive with centralized pools)?

For the sake of argument, suppose ALL mining had to be done through this built-in decentralized pool (or else the official clients wouldn't accept the blocks)... would it be possible or advantageous for traditional centralized pools to even exist on top of that? Are there technical reasons (other than perhaps being a pain to implement in a non-breaking and safe way) that such a change just wouldn't work?

Perhaps the two ideas could be used in combination (a protocol change along with a fund temporarily subsidizing miners or pools that adopt/support the change). I'm personally most interested in a technical solution to the problem - hence my posting to this board.

My only issue with a fund alone is that it seems to leave loopholes for people to game the system (e.g. pools staying "small" but secretly colluding). For that reason, I believe the solution needs to be incorporated into the software/protocol itself.

Thank you for your insights! It's a breath of fresh air (albeit a bit depressing) compared to the feedback given by people in the General Discussion board.

BTW - I agree with you with respect to levels of control <51% still carrying substantial risk. I mention the number "51%" because it seems like very few people on some of the boards actually believe that lower levels of control are still a significant threat, so without mentioning "51%", many people don't seem to understand what I'm talking about.


I realize that enormous amounts of money have been invested in mining hardware, and as you say, miners are all too often ignorant about the risks associated with mining pools.

But if you ask me, it would be better to try and fix this now (however painful as it may be - especially for pool operators who aren't going to essentially hand over their multi-million-dollar businesses without a fight), rather than suffer catastrophic problems down the road.

So if the problem is education and adoption (rather than technical), then I say bring it on! Save Bitcoin!


Thinking out loud here... if a protocol change actually enforced adoption of a P2Pool-like system, then I'm sure most of the pool operators would (strongly) oppose the change. But the ultimate power is in the hands of those with the hardware (who may or may not understand the need to adopt the change in the protocol). Aren't most miners familiar with the process of periodically updating mining software? What if an announcement were made - loud and clear - prior to the change, followed by the rollout of the updated reference client while urging all other client developers to update accordingly (perhaps there could be a cut over date/block index hardcoded in the software so that the change is somewhat coordinated)? If we could get a majority of miners to adopt the change (screw whatever the pool operators think), then the rest would have to adopt (or they'd stop getting any reward). We get companies like coinbase on board to help support the change by offering services that aid in the transition for current miners.

Thoughts? Is such a change completely out of the question?

In another thread (https://bitcointalk.org/index.php?topic=393815.0), we've been discussing the so-called "51% attack." The point I've made there is that regardless of your personal opinions about the profitability or motivation for such an attack, the vulnerability still exists and is particularly worrisome in the context of large, consolidating mining pools.

The original Bitcoin paper mentions the following:


My interpretation of the proof-of-work concept (reading between the lines) is that it is intended to distribute power among those who legitimately control and benefit from that hashing power. The problem with most mining pools in practice is that the pool operators can ultimately control the hashing power of pool members (obviously they don't have direct control of the the members' hardware, but with respect to the members' contribution to the blockchain, they do in fact have 100% control).

Putting it another way, I find it much, much less likely that the majority of the network's hashing power would be "dishonest" (willing to collude and carry out an attack) if each of the nodes were truly controlled by their owners as opposed to mining within a pool. Mining pools obviously offer some benefits to their members (over mining individually), but in effect, they undermine the goal of spreading the network's power over a large number of independent disinterested contributors.


So here's my 2-part question:

1. Would it be technically feasible to incorporate the concept behind P2Pool into Bitcoin itself? Obviously, it would require changes to Bitcoin clients and to the Bitcoin protocol. In effect, every mined block would be required to come from the decentralized "pool" and every contributor of hashing power would be rewarded - directly - for their share of contribution.

2. Do you think such a solution would help in reducing the chance of a 51% attack from occurring? Of course, I realize that due to the decentralized and democratic nature of Bitcoin, the possibility for a 51% attack would probably NEVER be fully eradicated, but perhaps by removing the utility of mining pools, the level of control currently held by pool operators would be redistributed to those who actually own/operate mining resources.


I would appreciate respectful and thoughtful responses the the questions above. Please, I'd ask you to refrain from making statements like "no rational pool is going to destroy/attack the network on which they rely to succeed". I'm not looking for arguments about why we "don't need a solution to the 51% problem". I'm interested in a solution that will help protect the democratic and decentralized objectives of Bitcoin regardless of whether you think these threats are rational, probable, or profitable.

You can just go right ahead with your blind ignorance. I respect your freedom to do so.


For those of us actually concerned about Bitcoin's longevity, how about giving some thought to my proposal earlier: what if we (and the Bitcoin devs) just built the concept of P2Pool (something similar) into Bitcoin itself? Basically, for any block to be accepted anywhere, it would have to be mined by someone participating in the one - and only - decentralized "pool".

If a system is only as strong/secure as its weakest component, I would consider mining to be that for Bitcoin. The way mining currently works encourages centralized mining pools (which happen to consolidate over time into large pools). This flies in the face of Bitcoin's core, foundational objectives.


The Bitcoin protocol is a piece of software. As a software developer, I can tell you that no piece of software is EVER perfect. I often get the feeling that people on these forums assume Bitcoin is perfect in its current state. That is not the case (the Bitcoin devs will be the first to tell you it's not perfect), but it's getting better every day.

So, why not fix this aspect of Bitcoin now before it becomes a problem? Why leave it up for debate, letting people speculate about how feasible it may or may not be? Why not just, you know, actually face these problems like clear-minded adults? I realize that changing the protocol (especially to modify it in this way) is no trivial task. But the whole concept of a decentralized cryptocurrency was no trivial task either, yet it exists (thanks to some hard work), and it's pretty dang awesome in my opinion. That doesn't mean we shouldn't try to improve it when weaknesses are identified (however "unlikely" we may think they are).

Instead of relying on people doing something "smart" and supporting a solution like P2Pool, is there some way the concept behind P2Pool could be built in to the Bitcoin protocol itself, disallowing any mining that doesn't happen in a decentralized way? In my opinion, the solution needs to be built in to Bitcoin at a fundamental level.

By the time you detect the block as being a "cheating block", the attacker will have already spent their coins and converted them into something that can't be taken back. All your fork does is restore the attacker's original balance, allowing them to repeat the attack again and again.

P2Pool seems like a good solution from what I understand. Yet according to blockchain.info, it accounts for only 2% of mining power. Can someone familiar with P2Pool vs other pools explain why it's not more popular? Is there something that could be done to help it grow?

You have got to be kidding me. Did you hit reply and quote me without reading what I said?

BEING "WORTH IT" DOESN'T MATTER.

If the attack is possible, then it CAN and WILL happen. Just give it time. If you are all really too blind to see that, then gosh, this community really isn't what I thought it was. I thought we (most of us) wanted Bitcoin to succeed. I thought we'd support a discussion about possible measures that would help ensure the long-term success of Bitcoin. Instead, all I hear is people saying "you're an idiot for thinking someone would want to do that."

Sheesh.

Believe me, I understand the idea of a distributed, peer-verified ledger. Perhaps you still do not understand that "being worth it" SHOULD NOT MATTER. The system is built to be mathematically strong, not strong due to someone's predictions regarding human nature and incentives.

OK. I think you guys are not understanding (or choosing to ignore) what I'm saying. I am NOT trying to argue exactly HOW a miner would make the attack profitable (I have no doubt that there are profitable methods for doing so, but that's not what this is about). I'm also not arguing that it wouldn't be obvious as to what happened after the fact.

It would be completely stupid to bring a gun into a bank and try to rob it. Any rational person would realize that the potential gain is not really very significant as most banks don't have a lot of cash on hand anyway. Any rational person would not risk the very real possibility of losing years of life, freedom, and "honest" income for such a low payout. A rational person would know that if you were to get away with it (such a low possibility), everyone would know the bank had been robbed. A rational person wouldn't attempt this, BUT IT HAPPENS ALL THE TIME.

The motivation doesn't have to make perfect sense for something to be a very serious threat. The >50% attack is known, there's little we can do currently to stop it (short of significant changes to the protocol and/or algorithms), and it's very much within reach for some of the pools.

Why in the world is there so much pushback (or willful blindness) about this?? Every day that Bitcoin becomes more mainstream, the stakes are higher, and the possible "exit strategies" for this attack become more plentiful. I sincerely don't want it to happen, but I guarantee it will happen eventually unless we are proactive about stopping it. And when it happens, we all (as supporters of Bitcoin) stand to lose.


Or, just go ahead and keep ignoring it because "it wouldn't be as profitable as you'd think." Because the chances are low (never mind the fact that a small probability multiplied over days, months and years becomes a very high probability).

I never said you have to wait for the 12th block to begin spending on transactions that will ultimately be nullified by the pool's private fork of the chain. You begin spending at 10001 (from your example), and you broadcast the private fork at 10012, recovering your spent coins. According to the wiki, with > 50% hashrate, the attack "has a probability of 100% to succeed. Since the attacker can generate blocks faster than the rest of the network, he can simply persevere with his private fork until it becomes longer than the branch built by the honest network"

Again, you don't have to wait - you spend, spend, spend, then magically reveal your fork that nullifies the transactions of the past 2 hours (or however long). Except, whoever sold you good/services in that time (or a middleman like BitPay) doesn't get to magically take back all that you stole.

And no, difficulty adjustments should have nothing to do with the scenario as they happen so infrequently, they really don't affect the outcome.


But still, why are we even talking about how easy it is "for the public to find you are trying to do this malicious thing", or the profitability of such a thing? It doesn't matter!

Let me put it this way:

If tomorrow morning you wake up and find out that some colluding mining pools have (surprise surprise) gone on a large-scale double-spending spree, stealing an enormous amount of value from merchants and service providers, which of the following questions are you going to be asking:

"How did this happen?" - No, because we already know how it will happen.

"Why did they do it?" - No, because who cares? I guarantee you won't be thinking "I sure hope the attackers were profitable in this."

"What can we do to prevent this from continuing to happen?" and "Why the heck didn't we take this threat more seriously?" - Yes, because even discovering who did it and what happened, you'll have little recourse. Sorry, can't whine about it to some central authority that makes everything right. You just have to suck it up (while the attack will likely keep happening, over and over) and try to actually do something about it.


But why wait? Why is this not discussed more seriously? Why isn't this a top priority issue?

So, it's my understanding that GHash recently mined 6 blocks in a row, with 25% of the network hashing power according to blockchain.info. Let's assume they collude with another similarly-sized pool and mine 12 blocks in a row (not unreasonable or particularly unlikely).

With an average time of 10 mins per block, that's 2 hours. I agree - not a LONG time in day-to-day life, but certainly a long time in the digital world. How much time (or how many confirmations) are most reasonable people waiting for these days? A few? Maybe 8 or 10 confirmations for something of high value?

Now, think of all the coins an attacker may have in reserve, and realize that they could easily all be double-spent in that somewhat short time period in transactions where people are being reasonably careful (waiting for several confirmations)... And unlike many other types of fraud, no one can simply reach in the attacker's bank account and seize/recover those funds. Bitcoin merchants & service providers will lose money, and they'll have little recourse.


Let's try some cost/benefit analysis: assuming 12 blocks of double-spend time... (we'll ignore the costs associated with operating the pool's mining resources as the cost will be the same in either scenario)

Opportunity cost to the pool: none if only the double-spend transactions are somehow reversed because you'll still get the block reward for those blocks. Otherwise, if those blocks are completely discarded by the network, then your opportunity cost is 12 * 50 BTC = 600 BTC (plus transaction fees).

Benefit to the pool operator(s) in attack scenario: The value of all coins in possession of the pool operator. Even after the attack is discovered and the double-spends are corrected on the network, you have whatever you spent your coins on + your original coins.

Risk: possible devaluation of Bitcoin if enough people become concerned with what you just did. But let's be honest - no one's going to care about a few double-spends, right?


I'm not talking about going out and "buying some weed" or having a crazy night on the town. There are plenty of businesses now (brick-and-mortar as well as online services and marketplaces) where people are willing to sell high-value items (e.g., exotic cars) for Bitcoin. Heck, many online exchanges only require 8 or 10 confirmations before funds can be traded. In that time, you could literally crash the market with your double-spent funds. Then just as people figure out what you've done, you buy back in as the market is recovering. Your double-spends are corrected by the network, but guess what - you still have all of your original coins + whatever you made while manipulating the market. As long as that's greater than the opportunity cost (accounting for the level of risk you feel it poses to Bitcoin's long-term value), then it makes economic sense to perform the attack.


Or maybe I'm missing something that prevents all of this... if so, please tell me.

But again, the debate really shouldn't even be about the profitability of the attack. It's about looking for real solutions to real problems (which can and will affect the future of Bitcoin), instead of dismissing them as being "unlikely" because "no one would ever want to."