Quote from: dashwhale on November 16, 2015, 09:11:59 AM

Quote from: GilAlexander on November 15, 2015, 09:39:56 PM

It's all clear about variance.
The weird thing is incremented "lastpayment" entry in the list without real payment occured. I believe updated "lastpayment" (from ./dash-cli masternodelist full, I observe it from 3 of mine vps) means that consensus decided some mn must be paid - and there're no ways to avoid that since the enforcement is active. Correct me if I'm wrong.
It's not about bugs in the monitoring scripts. Looks like dashninja.pl calculates "Last Paid=1d8h37m20s" based on "masternodelist full" output and dashwhale shows "Lastpay=6 days ago" based on the real payment transactions in the blockchain.
Anyway I'll inform about next payment. Wink

Yes, that's correct! Dashwhale lastpay status is derived from the blockchain, nextpay is derived from dashd. So your node moved from the 10% of masternodes, which will be paid within a few hours to the bottom of the payment queue without getting paid and without having been offline, which might indicate a rare bug. Since it's so rare, i suppose it's more worth to dedicate the core team's devpower to Evolution.

Actually, the way the masternode payments work is slightly different. Once your masternode moves into the 10% of masternodes eligible for payment, the system just uses some simple math to determine when you'll get paid. There's not a set time that nodes get paid, instead it's defined with probabilities.

Moocowmoo explains it really nicely here:

Quote from: moocowmoo on October 21, 2015, 12:36:49 AM

I think the math is correct here.

A node is eligible for for selection when 90% of the other nodes have already been selected.

Once you're due to be selected, your chances are 1 in 326 (10% of masternodes, the length of the selection queue) or about:
63% in the first 13 hours (326 blocks)
82% in the first 24 hours (576 blocks)
97% in the first 48 hours (1152 blocks)
99.5% in the first 72 hours (1728 blocks)
99.91% in the first 96 hours (2304 blocks)

where probability P = (325/326)^blocks

This will shift a little as the total masternode count changes, but the probabilities will stay close to these numbers.

Now, since other unpaid masternodes are constantly being swapped into the selection queue
and that selection depends on a competition between cryptographic hashes,
it is possible your node will fall OUT of the queue before being paid. (causing a skipped payment cycle)
This is a rare occurrence, but, double payments occur about 0.6% of the time, so over time payments should average out.

As moo points out, there is a possibility to fall out of the queue and not receive payment within a cycle. This only can happen when your node receives two votes for a given block and another node received more at the same time. This is extremely rare, as it would require a concensus disagreement among the masternode network; those are more common shortly after upgrading to a new version, then they quickly are solved and do not occur again but for very uncommon situations.

This is also not a bug, but will go away with evolution as well as the probability based design. I found another design that is much more memory efficient and isn't based off of probabilities, but instead will establish a set list of payees. I believe that's a lot better for the long term solution for us, so we'll move to that.