Quote from: jtoomim on September 18, 2017, 09:46:41 PM

Quote from: Cryptonomist on September 18, 2017, 11:06:32 AM

1) On the web interface of my node I find the following in relation to the shares: "Best share", "Verified heads", "Heads", "Verified tails", "Tails". I'm not sure I fully understand what they mean.

P2pool has a share chain that is very much like the Bitcoin blockchain, with a few differences. The tip of a chain (a block/share with no children) is known as a head. The root of a chain is known as the genesis block in Bitcoin (the block with no parent), but in p2pool it's known as a tail. P2pool's tails *do* have parents, unlike bitcoin, and consequently have a height greater than 1; however, they're called tails in p2pool because their parents have been pruned off from the node's copy of the share chain, so as far as your node is concerned, they have no living parents.

The best share is always a head. It's usually the head with the most work behind it, but not always. The rules for determining the head are a little complicated, and are different in different versions of p2pool.

Quote

*I suppose "best share" is the last share added to the sharechain (I'm I right about this???).

Not necessarily, but usually. Just because a share is recent doesn't mean that it is placed at the tip of the chain.

Quote

*"Verified heads" -- Other heads that are or will be orphaned or so?

Yes, precisely.

Quote

*What is the difference between "Verified heads" and "heads"?
*What are the "Verified Tails" and "Tails"?

I'm not sure. I think unverified heads or tails are shares that have not yet been connected to the share chain, such as for a share that is was just downloaded during initial sync but whose ancestors have not yet been downloaded. However, I haven't verified that hypothesis, as I have not yet needed to work on that part of the code.

Quote

*I suppose "Time first seen" gives the time that my node received a Inv ... Or is it the time when my node has completely downloaded the share?

Completely downloaded and mostly deserialized.

Quote

*"Peer first received from" gives then probably the node that has send my node the Inv containing the share.

Yes, but it's a sendShares or share_reply message.

Quote

*"Timestamp" is the timestamp from the node that has found the share I guess. But this timestamp probably depends on the accuracy of that nodes clock, and can in theory deviate from reality?

Correct. In the current p2pool master, the timestamp is required to be between 1 second after the previous share's timestamp and 59 seconds after the previous share's timestamp. Since the timestamps are used for minimum share difficulty adjustments, this can result in a situation where the share difficulty is very slow to adjust downwards after a rapid decrease in the network hashrate. In the jtoomimnet fork, I got rid of this timestamp clipping, and replaced it with a rule that rejects shares that are timestamped to come from the future in order to prevent share difficulty manipulation.

Keep in mind that the timestamp is the time at which the mining job was sent to the hardware, not the time at which the hardware returned the nonce value.

Quote

3) Say for example that I want to know the time it takes my node to receive the latest share of the sharechain (so I want to know the time it takes my node to converge to the consensus sharechain). I probably can use the timestamp of the share gegenerated by the node that found the share, as it will be very close to the moment that it sends it to the rest of the network.

Nope, it's not close. It will usually be off by around 5-50 seconds. But what you can do is use the "Peer first received from" line to browse to the info page for that share on the peer's address (you may have to guess the port number), and repeat, until eventually you get to a peer that says "Peer first received from: Null" for that share, and then you can use the "Time first seen" shown by that ndoe (or just the node that you got the share from, if you're lazy).

When I've done this measurement in the past, it's usually around 1 second per hop for nodes running CPython with 100 ms network latency, and around half that for nodes running pypy.

Quote

4) If my reasoning in point 3 is correct, I just need to find the time when my node is finished downloading the new share, and the share is added to the local sharechain. Is there a way to access the sharechain of my node directly through a log file or something? I know that p2pool creates logs in the directory /p2pool/data/bitcoin/, and occasionaly I find references to shares downloaded and stuff like that in the output. But is there another file for the sharechain, that contains time data of when a new share is downloaded and added to the share chain?

If you run my code (e.g. 1mb_segwit), you can install the rfoo package for python, then run run_p2pool.py with the --rconsole command line option, and then run the rconsole command in a separate terminal. This will give you a python interpreter that effectively runs inside the p2pool process, and allows you access to all of the variables and functions of p2pool while it's running. The share objects can be found in the gnode.tracker.verified object.
1mb_segwit also now has a --bench command-line option, which I think you would find interesting.

You could also maybe parse the log in data/bitcoin/log. That's just the stdout output of p2pool for (usually) the last few days.

Ok great. Thank you for the clarifications. I'll try the methodology you suggest and see if I can get the information I need.