That's actually a pretty smart idea, on the whole.

The problem is nova coin specifically, I know from mining on multipool it takes forever to get a block sometimes. For a pool this size, LTC can take a while too.

I think the variance is just too high for novacoin with our hashrate. We can go at it for hours, maybe a whole day and not get one block. Sometimes I just like to hop on middlecoin for 12 hours at a time, guess I can't do that anymore.

I am sure this will make some of you happy...

Mueslo - I finally just took your numpy.random.exponential and replaced my poisson function with it, in my own simulation. When I did, numbers started coming out looking like they should (actual return close to expected). This backed your claim that my distribution was wrong. As I started generating plots of each distribution, things started to make sense.

As hard as I've been dug in on this, most people would just bug out, but it wouldn't be right for me to not tell you that you were right on the distribution. H2O tried to explain this to me too, and a couple others.

My mental roadblock (or rather what cleared it) was the fact that even in a lopsided distribution you can still have an average, but it will be very skewed toward the thick end of the plot. I needed to actually see it plotted before it clicked.

But, my simulation was still correct (yours was too), less the distribution, so I guess I learned I'm better at programming than math :p

Anyway, thanks for actually trying to explain things to me and not being an ass like a couple others. I learned quite a bit researching all of this.


For the record:

<sub>import random
import numpy as np
import numpy.random as rnd

class worker():
    def __init__(self,hashrate):
        self.hashrate = hashrate
        self.sharesolvetime = 60 / hashrate
        self.shares = 0

class pool():
    def __init__(self,blockfindtime):
        self.blockfindtime = blockfindtime

pool1 = pool(30)
worker1 = worker(1)
worker2 = worker(12)
samplesize = 1000000

for n in range(0,samplesize):
    clock = rnd.exponential(scale=pool1.blockfindtime)
    clock1 = clock
    while clock1 > 0:
        sharesolve = rnd.exponential(scale=worker1.sharesolvetime)
        if sharesolve > clock1:
            break
        else:
            worker1.shares = worker1.shares + 1
            clock1 = clock1 - sharesolve
    clock2 = clock
    while clock2 > 0:
        sharesolve = rnd.exponential(scale=worker2.sharesolvetime)
        if sharesolve > clock2:
            break
        else:
            worker2.shares = worker2.shares + 1
            clock2 = clock2 - sharesolve
    
print "Worker 1 has: " + str((float(worker1.hashrate) / float(worker2.hashrate + worker1.hashrate)) * 100) + ' percent of the hash power'
print "But worker 1 has: " + str((float(worker1.shares) / float(worker2.shares + worker1.shares)) * 100) + ' percent of the profit'
print "Over sample size of " + str(samplesize)
print "When worker1's average share-find-speed was: " + str((float(pool1.blockfindtime) / float(worker1.sharesolvetime))) + 'X the block-find-speed'</sub>
    




Very Slow

Worker 1 has: 7.69230769231 percent of the hash power
But worker 1 has: 7.69363865886 percent of the profit
Over sample size of 1000000
When worker1's average share-find-speed was: 8.33333333333X the block-find-speed

Slow

Worker 1 has: 7.69230769231 percent of the hash power
But worker 1 has: 7.6898534448 percent of the profit
Over sample size of 1000000
When worker1's average share-find-speed was: 4.0X the block-find-speed

Medium

Worker 1 has: 7.69230769231 percent of the hash power
But worker 1 has: 7.68459728742 percent of the profit
Over sample size of 1000000
When worker1's average share-find-speed was: 2.0X the block-find-speed

Fast

Worker 1 has: 7.69230769231 percent of the hash power
But worker 1 has: 7.68286758249 percent of the profit
Over sample size of 1000000
When worker1's average share-find-speed was: 1.0X the block-find-speed

Very Fast

Worker 1 has: 7.69230769231 percent of the hash power
But worker 1 has: 7.67015222587 percent of the profit
Over sample size of 1000000
When worker1's average share-find-speed was: 0.5X the block-find-speed

A lot of you were totally sick of the diff conversation (totally understandable) and it continues safely away from your annoyance in the other thread.

There is one thing I was hoping to clear up though, based on the current conversation, and I want to ask here while we wait for H20 to confirm, because someone might know.

Does anyone know for sure exactly what reward system the pool uses?

Such as pay-per-share, proportional-per-block, PPLNS, etc?

Don't answer with a guess. Answer if you know.

The fact is I really don't know how it works right now. But, when I am talking about block change I mean block change for the network. I am starting to realize we are all working under a lot of personal assumptions about this because we don't know for sure what the reward system is. My assumption, and the one I've been arguing from (I hope not for nothing, but at least it would be settled), is that we get paid per block per shares that we have for that block.



This could happen, yes (very very unlikely but possible). But, you can simulate it.  That could happen in my simulation and I believe it would handle it correctly.

If my reward is set to 100, and neither worker got a share in time , it would simply pay 0/100 (0) of the reward to each.


We can continue to argue weather my system is correct in a proportional-per-block scenario, but we still aren't sure if that is the case anymore, so I'm just holding out to hear the scoop on that.

Part of me hopes I was dead wrong and H20 is using some other reward system so we can finally put it to bed.




It shouldn't effect the pool as a whole. The pool as a whole would still solve just as many blocks, it should only affect the shares within the pool.
 

I am curious what will happen if you implement the following conditions:

At the end of the block, worker shares get reset to 0.
Just before that happens, the shares are used to calculate payout, and the payout occurs per block.

Its not Miner A vs. the Pool and Miner B vs. the pool

Its Miner A vs. Miner B.

The distribution doesn't matter very much, because miner A and miner B both receive it the same system. I arbitrarily picked the averages out of the thin air. I can do that, because on the pool at any given time we can find miners all over the scale. That's why I can say miner A has this distribution, because no matter what, I can find a miner with that average.

To put it another way, its pointless to argue over weather miner A with average X would really distribute that way. Even if you are right, there is another miner with average Y who WOULD, and I can pick any of them since the competition is between the two miners, not miner vs. pool.

You were assuming straight PPS, I was assuming per block proportional. Do you see? We will never convince each other because we are both right, in the respective systems we think we are operating under.

Thank you for the detailed, thorough response. Give me a chance to digest everything you said. You are clearly more advanced in mathematics and statistics than I (i mean that genuinely not sarcastically) and I would love to learn from you. However, in the mean time, let me ask you this.

If you can't precalculate (you say precalculate, I say predict) the time is takes to solve a block, how is bitcoin (or whatever coin) doing it? Bitcoin itself has to predict the time it will take the network to solve a block, to know how to adjust the difficulty. It MUST maintain an average block find time of 10 minutes, or quite literally the entire thing will come crashing to a screeching halt.

If bitcoin couldn't accurately predict the time it takes to solve a block given X hashpower, we wouldn't be having this conversation because bitcoin would have died, and none of the alt coins that we love to mine at middlecoin would have ever existed.

Ok. I had a bit of an epiphany - the argument has been over what diff setting is correct this whole time, but now I understand we really should have been arguing over reward system.

Everything I am saying is accurate for a proportional reward system. If we aren't using a proportional reward system, woah, that probably should have been mentioned weeks ago.

What a lot of the arguments are saying would be true if there was one long block with no block changes.

That's the epiphany I had - we should be arguing to go to PPLNS.


The effect I am describing is present when each block has shares associated with it and when the block is found the reward is shared among those blocks. Everything resets for the new block. Proportional reward.

If we've been using something else, than no wonder we are all disagreeing, we are basically speaking different languages.

But PPLNS would eliminate the bias almost completely. The block change would then have no effect on the payout.

I will concede that in a PPLNS system, yes, all the diff does is introduce variance. I could modify my script in 5 minutes to simulate PPLNS and I bet the numbers would look good.



Lets change this conversation. H20, can you verify what reward system you are using now? It's not explicitly mentioned in your FAQ or announcement statement. Now I realize, unless everyone is on the same page on this we are going to spin our wheels forever.

Second, instead of arguing about diff - which would alleviate the problem I am describing but not eliminate it, can we talk about PPLNS or something similar?

Your example would solve something like, "what is the distribution of many hashes it should take to solve a share given 50% success rate".

What I am using poisson for, is to solve "given a known mean of 60 seconds, what is the distribution of time (in seconds) it will take to solve a share".

So the low, peak, low shape (skewed towards right for infinity) is correct for my use.



I can tell you'd prefer a full simulation instead of a half simulation. Mine is a half simulation since I am just using probability to say how long the block/share time took.

Let me give a full simulation some thought. I initially thought it would be hard to do, but the more I think about it, I think it might be possible to make each worker a thread. I thought there would be a race condition, but I forgot about the global interpreter lock in python. Basically python can't achieve true multi-threading because of the GIL which locks the interpreter to one thread at a time, but in this case, it would  actually be favorable.

Something like calculating how often a worker attempts a hash per second based on hash rate, putting a sleep(x) command in appropriate to that hash rate, and letting them all go to town until someone solves the block.

PPLNS is nice, but isn't that mainly just to stop pool hoppers? I doubt we have pool hoppers since it takes so long for coins to mature and exchange, probably a natural deterrent.

Are their other advantages besides anti-pool-hopping?

Alright. In the main forum, I posted my simulation script showing the skew towards fast miners. If you want the background, read that.

The main criticism was a fair one - I was "cheating" and using 0.5x and 1.5x of the average as the range for my random values for the share time, and block times. It was a shortcut that, admittedly skewed the results, but in such a negligible way that I felt it would still expose the effect. It was essentially cutting off the tiny sliver on the far right end the pierces out into infinity (plus some of the end on both sides). Its a minuscule amount of the distribution, but it was certainly a fair criticism.

So I did some research. The appropriate distribution is the poisson distribution. This is the same distribution that is used by bitcoin itself to calculate the expected difficulty for the next difficulty change. It is used in the prediction of the network, to maintain the statistical probability of one block every 10 minutes. Bitcoin would collapse without this prediction.

So it turns out theres a wonderful module in python called numpy. It has a built in poisson distribution random generator, making my life easy and making the change a breeze.

The main point about the poisson distribution, that should address the concerns: The Poisson distribution is not symmetrical; it is skewed toward the infinity end.

I also found and fixed a bug in my stats calculation. It didn't change the results enough to invalidate my previous conclusion, but these results should be accurate.

So with that, I reran my tests from before.


Slowest Coin

Worker 1 has: 7.69230769231 percent of the hash power
But worker 1 has: 7.29973071787 percent of the profit
Over sample size of 100000
When worker1's average share-find-speed was: 8.33333333333X the block-find-speed

Slow Coin

Worker 1 has: 7.69230769231 percent of the hash power
But worker 1 has: 6.86225439815 percent of the profit
Over sample size of 100000
When worker1's average share-find-speed was: 4.0X the block-find-speed

Medium Coin

Worker 1 has: 7.69230769231 percent of the hash power
But worker 1 has: 6.00872764122 percent of the profit
Over sample size of 100000
When worker1's average share-find-speed was: 2.0X the block-find-speed

Fast Coin

Worker 1 has: 7.69230769231 percent of the hash power
But worker 1 has: 4.23694576719 percent of the profit
Over sample size of 100000
When worker1's average share-find-speed was: 1.0X the block-find-speed

Very Fast Coin

Worker 1 has: 7.69230769231 percent of the hash power
But worker 1 has: 0.0129950864524 percent of the profit
Over sample size of 100000
When worker1's average share-find-speed was: 0.5X the block-find-speed

As you can see, the results are very similar.



And of course, obligatory code so you know I am not full of shit and can try for yourself.

<sub>import random
import numpy as np

class worker():
    def __init__(self,hashrate):
        self.hashrate = hashrate
        self.sharesolvetime = 60 / hashrate
        self.shares = 0

class pool():
    def __init__(self,blockfindtime):
        self.blockfindtime = blockfindtime

pool1 = pool(500)
worker1 = worker(1)
worker2 = worker(12)
samplesize = 100000

for n in range(0,samplesize):
    clock = np.random.poisson(pool1.blockfindtime)
    clock1 = clock
    while clock1 > 0:
        sharesolve = np.random.poisson(worker1.sharesolvetime)
        if sharesolve > clock1:
            break
        else:
            worker1.shares = worker1.shares + 1
            clock1 = clock1 - sharesolve
    clock2 = clock
    while clock2 > 0:
        sharesolve = np.random.poisson(worker2.sharesolvetime)
        if sharesolve > clock2:
            break
        else:
            worker2.shares = worker2.shares + 1
            clock2 = clock2 - sharesolve
    
print "Worker 1 has: " + str((float(worker1.hashrate) / float(worker2.hashrate + worker1.hashrate)) * 100) + ' percent of the hash power'
print "But worker 1 has: " + str((float(worker1.shares) / float(worker2.shares + worker1.shares)) * 100) + ' percent of the profit'
print "Over sample size of " + str(samplesize)
print "When worker1's average share-find-speed was: " + str((float(pool1.blockfindtime) / float(worker1.sharesolvetime))) + 'X the block-find-speed'</sub>
    



If you want to run it yourself, you need numpy. http://www.numpy.org/

If the block changes every 30 seconds on average, and you find a share every 30 seconds on average, and someone else finds a share every 5 seconds, how do you have the same chances?

You'll completely whiff on any given block just as often as you get one share.. meanwhile the other guy gets 6 in. If he has a little bad luck, he gets 5. You have a little bad luck? you get 0. But, it all evens out right? A little bad luck this time, a little good luck next time. Of course every time you have a little good luck, you don't get 2, you'll still get 1. He'll get 7

You each have a good block and a bad block. You have 1 share. He has 12.

1/12 is not the same ratio as the ratio as 1/6.

Because the block changed. We he finds one, its a whole new block by then. All his work on that last block will never be given credit.

Meanwhile, fast miner got 9 on the last block (or 8, or 10, whatever you like). Even if slow miner finds a block this time, fast miner probably got 8-12 shares again.

Over time, the block changing hurts the slow miner more.

When a block changes, every miner will have been working towards their next share.

But one share means different things for different miners. Losing one share for a slow miner is a bigger hit than for a very fast miner.

In the time the slow miner was working, getting nothing (yet), the fast miner already got work in, and got it accepted.

Higher ratio of recognized work / work actually done.

I thought about doing this. The problem is then I need to process the workers in a parallell fashion. Meaning run multiple threads/processes. This is because now both miners are racing each other. This complicates the code significantly, but also now you have introduced variables that you didn't have before, race conditions within the operating system.

I agree that would be a more accurate method, but to do it really properly you almost be better off just setting up a real pool and have real miners mine shares at different rates and take stats.

Minus my admittedly inefficient block/share time random generation (working on that), as long as you trust the probability it should be accurate, to the best of my knowledge. The block solve for each block is used by both workers on that block, so they can accurately be evaluated serially.

Edit: also, 2 workers is a really small sample in terms of solving a block, something that is in reality really consistent

I am in the process of updating my simulation script to include a more accurate distribution of block/share times.

I believe this to be a positive Poisson distribution at this time. I don't believe it will make much difference, but it needs to be addressed before some people will accept my work as proof.


For you new comers, the "con" argument to high diff is essentially this. Fast miners are more efficient at reporting their work, that is, they have less work go "unreported". Slower miners are less efficient at reporting their work, because the interruption of new blocks makes a larger percentage of their work go unrecognized. They will have been working longer without finding a share at block change.

Result: no change to overall pool profit. The effect is of the distribution if rewards.

The fastest miners are basically stealing some of the slowest miners hashrate, in terms of reward.

Now, I am no conpiracy theorist. However, it has crossed my mind that some of the people in this forum are some of the fastest miners. They might know about this effect, know it benefits them, and are therefore trying to keep the status quo. I am certainly not accusing anyone of that and I think it's unlikely that this is the case, just something to tuck away in the back of your minds.

I now think the correct distribution for my simulation would be the Poisson distribution. Can any math geeks confirm or deny this for me?

You are absolutely correct, that they are not between 0.5 and 1.5. But if i did between 0.25, and 1.75, 0.1 and 1.9, etc i'd get the same results long-term. And so on and so forth.

Accounting for the infinite right edge would be very, very difficult (not impossible) to simulate. In the end, it doesn't matter... because if I did, all that would do would shift the average somewhat to the right. But, the same thing would happen to the block time as well. They cancel each other out. At that point I am dealing with a different average. It doesn't matter what the average is that you use, the effect will still be there.

I am sure someone smarter than me can replace that small part of the code with a formula to simulate the real range. Its beyond my math knowledge. But know I am aware it is a flaw. I am almost certain the result would be very very similar if you did simulate that. I will attempt to do some research and see if I can figure it out. If i can, I'll re-run the simulations and see what happens. If anyone knows the right distribution for this, I can certainly convert it to python. H20 might be onto something about the inverse square distribution.

Again, I am picking arbitrary numbers to be the average. It doesn't matter what I pick. the point is the relationship between the two workers.

The block find time, and the share find time of the miner are going to be equally effected by any changes to the calculation of the times.

The following is my final word on this subject. I will shut up forever about it, after this. You are free to pick apart my script, find all the logical flaws with it, modify it, publicly shame/praise it. I don't care.

A lot of people aren't satisfied with theoretical equations. So I created a simulation. I wrote a script in python, it does the following things.

Takes 3 input variables.

• Average block solve time (a result of the pools hashrate and the network difficulty of the current coin)
• Worker 1 speed (your slower worker. represents the workers's hashrate in relation to the rest of the pool)
• Worker 2 speed (your faster worker. same)
• The sample size. This reduces the random variance over time if high

For each number of sample size, I run one instance of the simulation. Each one represents 1 block of a coin.

I generate a solve time for that block. Our constant variable is the block solve time, which represents an average. I generate a random number between 1/2 of it, and 1.5x of it, this is the solve time for this particular block. Remember, by the very definition of the word average, future values will be evenly balanced across both sides.

I then run a separate simulation, using the same solve clock, for both workers.

For each worker, I generate a random value which is 1/2 to 1.5x of their share solve time. Remember, these values don't have to be real, since all we care about is the relation to the other worker.

I check and see if the value is less than the clock. If it is, I credit the worker with 1 share, and subtract the share solve time from the clock time. I do this until the solve time finally becomes greater than the remaining clock.

Thus, I have simulated the number of shares that worker got from the block.

I do the same for the other worker, who has a faster share-solve-time.

The rest is just calculating and displaying statistics.

Heres the code:

<sub>import random

class worker():
    def __init__(self,hashrate):
        self.hashrate = hashrate
        self.sharesolvetime = 60 / hashrate
        self.shares = 0

class pool():
    def __init__(self,blockfindtime):
        self.blockfindtime = blockfindtime

pool1 = pool(500)
worker1 = worker(1)
worker2 = worker(12)
samplesize = 100000

for n in range(0,samplesize):
    clock = random.randint(pool1.blockfindtime/2,pool1.blockfindtime + pool1.blockfindtime/2)
    clock1 = clock
    while clock1 > 0:
        sharesolve = random.randint(worker1.sharesolvetime/2,worker1.sharesolvetime + worker1.sharesolvetime/2)
        if sharesolve > clock1:
            break
        else:
            worker1.shares = worker1.shares + 1
            clock1 = clock1 - sharesolve
    clock2 = clock
    while clock2 > 0:
        sharesolve = random.randint(worker2.sharesolvetime/2,worker2.sharesolvetime + worker2.sharesolvetime/2)
        if sharesolve > clock2:
            break
        else:
            worker2.shares = worker2.shares + 1
            clock2 = clock2 - sharesolve
    
print "Worker 1 has: " + str((float(worker1.hashrate) / float(worker2.hashrate + worker1.hashrate)) * 100) + ' percent of the hash power'
print "But worker 1 has: " + str((float(worker1.shares) / float(worker2.shares)) * 100) + ' percent of the profit'
print "Over sample size of " + str(samplesize)
print "When worker1's average share-find-speed was: " + str((float(pool1.blockfindtime) / float(worker1.sharesolvetime)</sub>
    

    

It displays the following stats:

• What percent of the hash power worker1 has
• What percentage of the profit (shares) he ended up with
• What sample size we used
• What was the ratio of worker1's time to find a share to the pool's time to find a block (another way of saying, how "fast" was the coin)

I will now give you the results of running this script. I will use the same workers speeds, but I will change the block solve time. I will give 5 examples

One important point - the block solve time represents the speed of the coin, we can't do anything about that
The share solve time - that's what we want to effect. Two ways to do this: change your hashrate, or change the share difficulty.

Now then... the results...


Very Slow coin (something like LTC):

Worker 1 has: 7.69230769231 percent of the hash power
But worker 1 has: 7.12127534135 percent of the profit
Over sample size of 100000
When worker1's average share-find-speed was: 8.33333333333X the block-find-speed

Pretty Slow coin

Worker 1 has: 7.69230769231 percent of the hash power
But worker 1 has: 6.6950187416 percent of the profit
Over sample size of 100000
When worker1's average share-find-speed was: 4.0X the block-find-speed

Medium/Fast Coin

Worker 1 has: 7.69230769231 percent of the hash power
But worker 1 has: 5.89708931026 percent of the profit
Over sample size of 100000
When worker1's average share-find-speed was: 2.0X the block-find-speed

Fast Coin

Worker 1 has: 7.69230769231 percent of the hash power
But worker 1 has: 4.07045734716 percent of the profit
Over sample size of 100000
When worker1's average share-find-speed was: 1.0X the block-find-speed

Extremely Fast coin

Worker 1 has: 7.69230769231 percent of the hash power
But worker 1 has: 1.15306809456 percent of the profit
Over sample size of 100000
When worker1's average share-find-speed was: 0.5X the block-find-speed

A quick analysis of the results supports the following conclusion:

There is a skew towards faster miners, in terms of their percentage profit compared to their hashrate, on any pool for any coin. The effect gets exponentially increased as the time for the worker to find a share comes closer and closer to the pool block find rate.

This effect is negligible, and irrelevant for slow coins who have slow block time. However, as share-find-time for slower workers approaches block-find speed for fast blocks, miners begin to lose an extreme amount of profits.

This only takes into account block changes, where the client hears about the new block - there are also rejected shares, where it hears about the new block too late, and happens to solve the share before hearing it (this is what is measurable on the site's stats page), and also coin changes.