remember that are not the miners that make the fee, they could accept even the lowest fee if everyone was using the lowest one and reducing the average, it's up to the peple that decide what fee to use
if for example everyone start now to lower his fee the average would be like the old 10k satoshi per transaction, but instead everyone is rushing to have his/her transaction prioritized and this is increasing the average, miners have no fault here
I'm not so sure of that...
I mean why would the miner not simply agree to not accept a transaction with fee lower than X
or even do the transaction while putting fees themselves to make higher average fees...
I've seen blocks not full at all, doesn't it mean that miners are refusing transactions in some cases?
Since there is no agreement between all miners and there is no agreement between all Bitcoin users we will always have both types in each party: miners who will accept lower fees and users who will agree to pay higher fees, like 400+ satoshis/byte right now for example, to get their transaction confirmed fast. And that's a good thing IMO, because every transaction will be confirmed eventually and if you want your transaction confirmed fast, you know what to do.
Having said that, I would not recommend to use fees lower than 120 satoshis/byte nowadays.