Quote from: windpath on June 20, 2014, 02:28:17 PM

Code:

This is a great conversation guys, and one that needs to be had.

I have built and managed large development teams in the past for specialized projects, and I have to say that none of them came close to the complexity and challenges offered by what we need in a p2pool dev team.

For p2pool to get to the next level I think we need an active lead dev or chief scientist to lead; and 3 technical achievements, the first 2 are easy:

1. Hardware compatibility and a responsive available individual for both HW and mining SW folks to work with.

2. A great looking front end that provides what miners want that is unified across nodes. p2pool already has the ability to be the most transparent pool in existence, lets get that data out there for miners to mull over and do it in a way that looks so good out of the box that node operators are not inclined to re-invent the front end with each install. When a miner checks out a node it should be familiar and they should know what they are looking at. I'm happy to build this using open source tools and have a solid start already on my node. Combining Bootstrap, PHP, and an SQL DB we could build an open source, cross-platform, front end that both looks great and is data rich.

Which leads me to our 3rd, and biggest challenge:

[b]3. Vertical scalability that reduces variance for all miners.[/b]

This is where the real challenges lie.

For p2pool to support large mining operations, and still be able to attract medium, small, and micro miners we need completely new share difficulty and payout structures.

The first step here has nothing to do with code, some completely new concepts need to be developed and generally accepted by p2pool miners. Once we have those concepts down, and can demonstrate them to be technically possible, we can seek out devs with the chops to pull it off.

This may seem like no big deal, but I assure you it is. Here is my first crack at it:

[b]Payouts[/b]
While miners meeting a certain threshold could still be paid directly from the generation TX, smaller miners under the payout threshold when a block is found, lets say [btc]0.01 to start, would be able to see their p2pool earnings down to the Satoshi in real time, but would not receive a payout until reaching the threshold.

To accomplish this I propose a decentralized, trust-less escrow system.

A p2pool software controlled secure wallet where all payouts from each block that did not meet the payout threshold would be stored, and then paid out to miners when they reach the threshold.

Miners who reach the minimum payout during a round are paid directly from the generation tx, while payments for all miners under the threshold are are sent to and stored in the wallet, and payouts are made from the wallet when the threshold is reached.

The caveat is that this needs to be handled entirely by p2pool in a decentralized and trust less way with no single "admin" overseeing them. How to handle the keys securely without anyone but the p2pool code knowing them is the challenge, maybe the guys from the dark wallet or Armory teams would have some technically viable suggestions.

[b]Difficulty[/b]
An automated "variable and weighted" share difficulty would need to be assigned to miners based on hash rate. Every major pool in existence does it, we just need a method of doing it across the p2pool block chain in a way that is not easily manipulated. Perhaps the MIT Bitcoin club would be interested in tackling such a challenge?

[b]End game:[/b] Protect the network, encourage miners to participate actively in both Bitcoin and p2pools decentralized nature.

Even without active development p2pool is a strong "brand" in the Bitcoin space, most miners are aware of it, and its challenges. Overcoming its biggest challenges along with a decent marketing push (by us collectively) spreading the word could easily push p2pool to the top and keep Bitcoin strong and trust less for the future.

Just my 0.02 bits.

I think the biggest obstacle with your proposal is:

Quote

A p2pool software controlled secure wallet where all payouts from each block that did not meet the payout threshold would be stored, and then paid out to miners when they reach the threshold.

I'm not sure the concept of a secured wallet that needs to maintain thresholds for users and process payouts dependent upon when a user crosses that threshold can be translated into a decentralized model. Centralized pools like GHash/BTCGuild/Eligius/Slush all have their own payout models and can pull this off because they are indeed centralized.

We need a way to reduce the variance that will inevitably occur as more miners are added to the p2pool network. As I previously wrote, this is what I believe is holding p2pool back from becoming a more mainstream option. Think of it like this: if every single BTC miner decided to join the p2pool network that means ~100PH/s and share difficulty would become 1/20th of the current BTC block difficulty.

Code:

Current BTC difficulty: 13462580114.5253
Divide that by 20: 673129005.726265

A share difficulty of 673.1M is not a feasible model. Assuming I have 1TH/s:

Code:

Difficulty * 2**32 / hashrate / 86400 = number of days to find a share
673129005.726265 * 2**32 / 1000000000000 / 86400 = 33.46142437017714

Over a month's hashing at 1TH/s to expect to find a single share, assuming the difficulty remained constant, which it wouldn't, because BTC difficulty is going to adjust every 2016 blocks. You're looking at needing 12TH/s or more to expect to get a share onto the chain within the 3 day payout window. Nobody besides the "big guys" currently has that kind of hashing power. The closest you're going to get will come in August when Spondoolies-Tech starts shipping their 6TH/s SP30s. If you happened to get in on RoadStress' group buy around Easter time, 2 of those would have cost you about $9000 USD.

Of course, the example I gave is completely contrived. Nobody expects and/or believes that p2pool will become the sole BTC mining operation in existence. Maybe we do nothing at all, and just accept the fact that unless you increase your own hashing power, you will experience more and more variance as your hash rate becomes a smaller and smaller portion of the p2pool network's rate. As I wrote in my first reply on this topic, the current entry level hardware to p2pool is pretty much an Antminer S1. Soon, that entry level hardware will be 500GH/s (the Antminer S3). Perhaps this is just the natural progression of things and changing the underlying way p2pool works to accommodate the smaller hash rates is futile. Even if we go with a multi-tiered approach, as I suggested, how do we define those tiers? What becomes the "cutoff" value to move from tier 1 to tier 2 to tier n? How many of these tiers would be "enough" to effectively reduce the variance?