To me, it seems anything but negligible, and seems like common sense.
The bigger block, the longer the time is required to process
it, and the greater the the risk of orphaning.
What am I missing here?
All of the expensive operations can be done before a block is found, then they do not add any proportional time.
Alternatively, miners can just consolidate to larger and larger pools-- which don't suffer any of those delays between themselves. Historically, this is how miners have responded to higher orphaning.
Alternatively alternatively, pools can just implicitly trust each other's work-- this is functionally equivalent to becoming a single larger pool for the purpose of this discussion.
I hope all this hostility isn't getting to you, Gmax. Keep fighting the good fight.