The problem is the baseline and how much of a difficulty change equals how much of a block size limit change. The baseline could be how much difficulty is at the time of the change (for example), but the other part is tricky. Maybe it could be done as a percentage change, it would change the same percentage as difficulty?
This is what I mean when I say that an oracle would be needed to determine the formula relating difficulty to block size. There is absolutely no way to know what this formula would look like. The penalty for getting it wrong (and it would be wrong most of the time) is vanishing fees, resulting in hysteresis (wild oscillations in network hash rate). ASCIMiner's 24% of the network hash rate could become 51% after a difficulty adjustment.
What's wrong with the scheme I described, which responds directly to changes in transaction space scarcity?
If we...use soft limits and/or other block verification rules to impose scarcity on transaction processing, most of the miners & pools will abide by the soft rules even when commenting out those rules is provablely in their own economic interests. Reputation matters here, even moreso than it does in the "real" business world.
False. At least one of the pools will rewrite the soft limit to their economic advantage. Once this happens, members of other pools will quickly abandon ship and join the pool that modified the soft limit, since they have higher payouts. More realistically, all the competitive pools would make this change immediately after getting a source code patch with a soft limit. The only reason you don't see it happening now is because the block reward subsidy is orders of magnitude greater than the fees. As the block rewards get cut in half, there will be increasing pressure on miners to optimize their selection of transactions and that means abandoning the soft limit.
You're saying that miners will choose to make less money rather than more? Miners at the margin (those who would go bankrupt with the soft limit) obviously will choose to optimize the transaction selection code rather than going out of business. Your premise that reputation matters more than profit is wrong.