Quote from: harding on July 09, 2020, 05:37:39 PM

However, based on these data, I don't personally agree that "there isn't much activity in improving the Bitcoin protocol anymore." The rate of change does seem slower, but I think it's far from stagnant.

I think your post supports my position.

A change in the P2P protocol would require or at least enable a behaviour change by a peer. Only one of these PRs do that.

Almost all those commits in the recent change are implementation optimizations or refactoring, they don't change the P2P protocol-- many don't change the externally visible behaviour at all.

I would generally characterize more recent activity as being extremely biased towards making changes that appear have generally no effect (if not absolutely have no effect), which allows people to feel more confident that they don't break anything. This isn't to say that I don't think the changes are good, I think almost all of them are, but I would also say that only a couple of them accomplish anything a user might care about.

#19219 -- Optimization. Corrects an exploitable vulnerability reported by a bcasher, took over a month to get merged. Doesn't change externally visible behaviour except for the vulnerability (and extremely rare hash collisions). #19204 -- Local behaviour change that doesn't change the protocol. (proposed in a slightly different form in 2016, but never implemented in that form) #19260 -- Straight up revert of #8709 which would mess up compatibility for nodes which implemented the 2016 alternative to #19204 (which presumably no one did, so it's not a problem) #16939 -- Local behaviour change. If addrman isn't empty wait 5 minutes before querying DNSseeds instead of 11 seconds. Only a behaviour change at all when getting a connection takes longer than 11 seconds but less than 5 minutes for a previously run node. #19010 -- This is part of an actual protocol change, though one that is optional, disabled by default, and somewhat resource expensive and described in a 2017 (three year old) BIP. (also, not obviously useful-- though that's my opinion). #18960 -- implementation optimization to facilitate #19010, no externally visible behaviour change. #18861 -- local behaviour change. I ran a node for the 2 months with a modified version of the PR that logged whenever it was triggered. It was never triggered, so in practice it's not even a behaviour change currently just theoretically. (I also got sipa to run my patch on his node, with the same results) It's a fine enough change and I was happy to see it, though the leak it prevents still exists in other forms. #18962 -- local behaviour change that only matters against a non-existing conjectural alternative implementations. #18877 -- part of #19010 #18808 -- local behaviour change, doesn't make any difference with existing implementations. (Also I don't understand how it philosophically meshes with other changes from the last year or two which increase the nodes aggressiveness in disconnecting on unexpected messages, such as #19260, #15759, etc.) #18038 -- local behaviour change to be much less aggressive in rebroadcasting transactions that haven't been mined in a long time and won't get mined soon. Adds separate tracking so that it's still aggressive about resending things which have never been getdataed at all, so that the reduction in aggression doesn't result in failing to broadcast entirely. (Yippie! I hadn't seen this had gone in, I'm particularly happy about it.) #16902 -- implementation optimization, no externally visible behaviour change #17951 -- Extremely narrow local behaviour change, more of an implementation optimization. Uses a cheaper but differently imprecise mechanism to avoid requesting broadcasts of already confirmed transactions (e.g. this one won't ignore long-ago-txn-with-still unspent outputs and has (rare) false positives). Though it isn't mentioned in the PR anywhere that I can see, I believe that this change is significantly motivated by a future plant to change to witness-ids for relay: there is no index on nodes of past witness IDs, so the older check wouldn't work at all in a post-witness-relay world. If that's correct, then this could be seen as a facilitating change for a future protocol change. #16702 -- Experimental feature, off by default, requires a 'map file' that was difficult for me to produce as a subject matter expert. Basically in places where the software treats /16 peers as being in the same network this replaces that with an ASN database. Perhaps when it becomes non-experimental and the project figures out how to ship a map file I could upgrade this one to local behaviour change. (It's a pretty cool local behaviour change, but still just a local behaviour change).

So there is only one thing that I would consider an actual change to the P2P protocol, and it's the realization of a protocol written in 2017.

In the context of this thread-- the question was observing stale blocks. The idea of adding a P2P message to allow relaying their headers has been suggested many times probably going back to 2012. It came up more often post 2017 or so when it became clear that block relay improvements were making it extremely hard to observe orphans.

But no one is working on that. Now, if it had happened and been in your list I would have described it as an unimportant feature that wouldn't improve life for end users (though it would significantly improve developers ability to gauge the health of the network)-- given that and as you sample there are have been few recent changes that change the protocol, I really don't expect it to happen soon unless changes.

Things aren't completely stagnant, e.g.

I've seen some activity towards revising ADDR messages. The current addr message are incompatible with v3 TOR hidden services, which have existed since 2017 and have much better properties. If this work is not completed and deployed within a year hidden services will stop working for Bitcoin users. Given the normal release and deployment timelines as well as the fact that the revised version is not merged or apparently merge-ready, I think it is now extremely likely that onion-peer support will be disrupted for users, in practice. Work on this began two years ago but didn't make progress for a long time.

There has also been some activity towards wtxid based transaction relay and the related erlay change.

But these changes are all moving extremely slowly, to the point where-- personally-- I forget everything about them between updates to them and don't find it interesting to contribute to them. Especially as I've found the testing framework makes it extremely tedious to work on any P2P behaviour change, because behaviour changes tend to break a wide assortment of unrelated tests, that people are slow to review or comment on them, and that they're continually broken by changes which are fast moving because they don't actually change behaviour.

Given that there isn't (yet) a fire behind p2p protocol updates required to keep onion peers working. I don't think it's unfair to characterize things as stagnant from the perspective of the p2p protocol itself and I really can't see purely instrumentation changes like stale-header-broadcast happening any time soon.