concern of a "downward spiral of transaction fees"?
If blocksize isn't scarce at all why not include every transaction that pays at least 1e-8 BTC which you've already received and validated? If it's purely orphaning that influences your decision why won't all your spending go to bandwidth (neutrino links to other miners and such 
) and none to POW? etc.You gotta limit blocksize at some level. Otherwise it's probably just too easy to release ridiculously large blocks which fork the blockchain.
That's what I used to think too. However, the system is self-stabilizing, ridiculously large blocks which fork the blockchain will find themselves orphaned. If unlimited size is too worrying then the simplest safety net would be max block size = 10x median block size during previous 2016 blocks or previous difficulty period.