Thanks for the reply, forrestv. It's always good to see your text.

I didn't think about it from a selfish mining perspective. Interesting. I don't think we're at the Nash equilibrium, though.

I agree that C2 is more helpful to the pool than B1, and ceteris paribus, I think that for the sake of the pool it's better if people follow C2 instead of B1 when the choice is forced; however, I do not agree that a self-interested miner would always choose to mine on top of C2 instead of B1 when given the choice. Let's say both D2 (based on B1) and E2 (based on C2) get mined. Which one gets preferred?


(In this diagram, the letter refers to the person who mined it and the number refers to the Bitcoin block height.)

Note: The section below was heavily edited a few hours after posting.

At the point shown in the diagram, neither of the valid heads (D2 and E2) are block-stale, so neither one gets punished for that. Ultimately, the share that wins is the one at the greatest height (edited: not the most work), or the one that was propagated first if the heights are the same. As such, there is no direct incentive to guide miners to prefer to mine like D2 versus like E2. As for indirect (pro-social) incentives, some miners might prefer to punish shares that are block-stale, whereas other miners might prefer to punish shares that are share-stale and which may be selfish mining attempts. Personally, I'm a lot more worried about selfish mining attempts on p2pool than I am about block-stale shares, so I personally would prefer to make or build off of D2 rather than E2. But let's say miners are split 50/50 on that issue.

Since there's no clear incentive why a miner would prefer to make E2 instead of D2, C2's choice to mine on A1 instead of B1 comes at a substantial cost. If D2 has already been mined and C just hasn't heard of it yet, then C2 will be at a substantial disadvantage. C2 will always lose against D2 if C2 is based on A1, but C2 has a roughly 50% chance of winning if C2 is based on B1. As we know that B will be trying to mine on top of their own blocks even after C2 is mined, it is clearly disadvantageous for C to try to orphan B1 if B has more hashrate than C and if bystanders are indifferent. It is also disadvantageous if bystanders prefer C2 but B has more than 50% of the hashrate -- which, as it happens, is currently often the case.

So C loses if D2 is in flight (maybe a 10% chance) or if B mines the next block (maybe a 10% chance); or if other miners don't clearly punish block-stale before share-stale (maybe a 50% chance). P(C_wins) is around 0.3 to 0.8. If C wins, the benefit to C is one fewer competing share in the share chain (maybe worth 10% of a share if C has 10% of the hashrate). Thus, even if all miners punish block-stale shares first, there's still roughly a (1 - 0.8*110%)=12% cost to the current strategy; and if the punishment strategies are 50/50, then there's a (1 - 0.3*110%)=67% cost to the current strategy. (End of edits.)

By choosing to base C2 on A1 instead of B1, the miner of C2 is gaining a small benefit from orphaning a competitor's share, but is paying the cost of C2 having lower absolute work. This generally works against C's best interest. A self-interested (but not large and maliciously selfish) C always acts in their own best interest when building off the chain head with the most work.

Agreed. However, high orphan rates worsen the UX in a few ways: (a) they increase reward variance; (b) they confuse people who don't understand that it's only the difference in orphan rates that affects revenue; and (c) they can be pathological in some edge cases, such as when the pool hashrate drops dramatically (e.g. during my hard fork) and the time between shares is greater than or comparable to the time between blocks, or for altcoins with naturally short block intervals.

I also think that creating orphan races results in greater reward unfairness, since winning an orphan race depends unfairly on miner hashrate and bandwidth, whereas sequential mining does not. While I agree that the stale-block rule would have the same chance of putting any miner's share into an orphan race, I do not agree that it gives every miner's share the same chance of being orphaned.

So looking back at your list of rationales:
I disagree that it's the optimal strategy or a Nash equilibrium, so I think it's avoidable.

I disagree that it's fair, as large miners are more likely to win an orphan race.

I'm on board with incentivizing fast bitcoind instances. But maybe there's a better way? Rather than a stick, why not a carrot? We could give a small revenue bonus (or lower difficulty) to any share that uses a block with more total work than the block of the share's parent. The current strategy seems to me to have more drawbacks than advantages, but I think a carrot wouldn't have any major drawbacks. Edit: a carrot incentivizes people to intentionally orphan the first share on a new block. The perverse incentive might be weak enough to never be worthwhile, but it would require some care and may not be worth the complexity cost.

Which means that with the current code, big miners are playing to win when it comes to orphan races, and small miners would be wise not to start an orphan race with a large miner. Given that we currently have one miner with around 70% of p2pool's hashrate, this makes this block-stale punishment strategy suboptimal in practice, not just theory.