No one is talking about anyone demanding a refund, but whether it would be a moral thing to do from the recipient's perspective.
It would of course be the right thing to do from the sender's perspective. But is it the right thing to do from the individual miners' perspective? They have honestly mined a valid transaction, and now their pool operator is denying them the profits? Should it really be up to the pool operator at all? Should each individual miner not get to decide what to do with their share of the excess fee which they earned honestly?
I don't think it's as simple as "I paid too much in fees, give it back".