if there is an "global optimal max size", it's quite pretentious to claim you can come up with the "optimal formula" to calculate it.
Definitely, but I was talking about an optimum strategy for prioritizing transactions, not an optimum choice of max block size. Typically the strategy will be either:
1) Include all known pending transactions with fees
or
2) Choose the pending transactions with the highest fees per kilobyte ratio and fill the block up to a certain size
There would be some variety, surely. In the blocks they produce themselves, miners will search to optimize the ratio (time to propagate / revenue in fees), while in blocks they receive from other miners, they would rather it be the smaller possible. These parameters are not the same for different miners, particularly the "time to propagate" one, as it strongly depends on how many connections you can keep established and on your bandwidth/network lag.
I don't understand this aspect of the network. Why do miners want smaller blocks from other miners? Do blocks take a long time to propagate? Are you saying that newly solved blocks are sent around on the same peer connections used to transmit messages, and that while a connection is being used to send a block (which can be large relative to the size of a transaction) it holds up the queue for individual tx?
If this is the case, perhaps an easier way to deal with the propagation of blocks is to have two overlays, one for tx and the other for blocks.
I think that if we reach the 1mb limit and don't upgrade with a solution, then the spontaneous order will create fiat currencies backed with bitcoins, in order to reduce the amount of transactions in the bitcoin network.
I'm not so sure this is a bad thing. These ad-hoc "fiat" currencies may be created with unique properties that make them better suited to the task at hand than Bitcoin. For example, a private payment network that provides instant confirmation and requires no mining (relying on trust in a central authority).
So, this would also lead to less revenues for the miners (plus a loss on reputation for the bitcoin network).
Average transaction fees per kilobyte is inversely proportional to the block size, so leaving the block size at 1MB will cause fees to increase once blocks are regularly full. The rate of increase in the fees will be proportional to the growth in the number of transactions.
Miners would love it if all blocks had a one transaction maximum, this would maximize fees (assuming people didn't leave the Bitcoin network due to high fees).