So then, a reliable metric for node operation is to keep track of the information being broadcast to the network. Since it's the capabilities of the network that we care about, maybe there would be a simple integer appended to the information broadcast to the network indicating if the node is below or near it's peak capabilities? If every node attached this information to the messages it transmits to the network, and then it was read as an aggregate from the completed blocks, it would only add 2 bytes to the minimum size for a block and then the block as a whole could be read to determine the average. The information transmitted is also an aggregate of various factors applied to the machine, to give an average which ends up being more of a vote.
So, as an example: Suppose we have 10 nodes participating in a specific block. 6 of them report a 0 - indicating that they are well below capacity. 3 reports a 1 to indicate that are near capacity. 1 reports a 2 indicating it is at it's limit. That means we have 6 votes to increase the cap, and 4 votes against, but only IF the current block size equals the current block size limit. If they are full, the next block has a slightly higher ceiling. The amount of which could be based on an additional digit within that integer as some signal to the network of how much more it can handle. The consequence is that the node which is at capacity won't be able to participate in as many transactions, so will get fewer votes.
That is a simplified example since a given node might have thousands of transactions that it has participated in, but that's OK. If it handles more transactions it has more votes, meaning that smaller nodes that are not as capable might participate in less transactions given it's self identified capability rating. Giving node operators who have a wallet attached to the node a piece of the transaction fee for participating in this process would provide incentive to a node operator to upscale to handle more transactions.
Now, if the reverse happens where a majority of node operators vote that they are at capacity, the network might cap there, or if the votes are leaning towards it being too much the network might scale back - even if the block isn't full. A formula could decide how much to scale it back as well.
I don't see any reason why we can't do this.
Any metric that requires self reporting and cannot be independently verified to be true is easily gameable. Suppose I really want larger blocks. What prevents me from spinning up a few thousand fake nodes (you can't tell they are fake, they all look real as that is just the nature of nodes) and then having each node vote saying that it is super overloaded and we need Gigabyte sized blocks now? What prevents me from lying? What if I am actually experiencing no load at all but I claim to everyone else that I am absolutely overloaded? There is no way for you to verify that what I am saying is true or now without having direct access into my machine. Are you really going to trust what I say and take it at face value?
That doesn't mean there's a limit to how long the channel can remain open though, so in cases of people who leave it open indefinitely that ends up being even worse for everyone else who uses on chain transactions compared to those who occasionally close the channel.
Why would it be worse for everyone else? They aren't making transactions on chain, so no one else is effected.
In general, changing what the block size limit is requires considering more than just transaction fees and capacity. You also have to consider things like the potential for increased orphan rates, the potential for fee sniping, whether nodes can support the larger block size, any potential attack surfaces or ways that different block sizes can exacerbate current issues (e.g. quadratic sighashing), etc.
Lastly, a meta note. Please don't make so many consecutive posts to respond to each post individually. Do them all in one post. You can separate each response using a horizontal rule as I have done here.