There _must_ be competition for block space to make fees a viable way to pay for security.
Limiting block space isn't the only way to increase mining fees. Removing/reducing the block reward has the same effect. Once the reward is less than the cost to mine, most miners will require a tx fee. Given this incentive, there's really no reason to limit the block size. It is in the miners best interest to set the limit as high as possible so they can receive the maximum amount of tx fees possible. For this reason alone I foresee the block limit rising significantly.
It doesn't really work this way. It costs nothing to include a transaction so a small, self-interested miner will include every fee-paying transaction, even a small fee. In so doing, he sells out the bargaining power of miners; users will see that low-fee txs are included, and will pay less fees in the future. The damage thus done is shared by all miners, but the profit from including the transaction is all his own. This is a
tragedy of the commons problem. The common resource here is the bargaining power of miners, which the individual miner will consume at others' expense.
This will be alleviated if mining is highly centralized (which we don't want), or if there are other factors such as assurance contract funding or a limit on total bitcoins transferred per block.
Why do you believe this (that small self-interested miners will survive)? Why must this be a tragedy of the commons? What is special about bitcoin mining that makes block space the only scarce resource that cannot find a price? Removing the limit doesn't make it a non-scarce resource, as precisely the fact it is a scarce resource is what is apparently scaring so many commenters.
My gut feeling is that the block size limit should be removed, and let the free market reign. It works elsewhere, why not for transaction processing?
You're effectively claiming that some miner can survive and be profitable / just eat the cost by being selfish and adding all transactions no matter how small the fee. Good for her! She must be more efficient than everyone else. If you can't stand the competition, stop trying to compete and get out of the race. If mining is not profitable, miners will drop out until it is profitable, just like happens now. Everyone will benefit from the competition towards more efficient mining setups and lower transaction fees.
The argument that block size must be limited seems to me to ignore that:
- every node is free to not relay "spam" blocks, in the same way they are free to not relay spam transactions now, for whatever definition of spam that node sees as fit.
- hashing transactions into blocks has a cost, as does relaying large blocks. If your block is "too big", for some technology and market-determined definition of big - then you will be beaten to the relaying race by a smaller, lighter, more network-acceptable block.
- a real feedback loop exists here to determine the right pricing and policies, "on average"
- every node is free to implement whatever policy on this stuff they want.
- if sufficient hashing power believes a certain miner or pool is an anti-social PITA, they can put "social pressure" on them by not relaying their stuff.
Remove the block size limit entirely I say. Ultimately it is effectively just another indirect input to block difficulty, and we let all the other inputs such as CPU/GPU/ASIC technology advances float, and so we should do the same for block size.