How does a minimum difficulty of 512 make a pool not "fair"?
Because I would submit more shares on lower difficulty. Or am I wrong on this?
Every single pool that uses difficulties greater than 1 (AKA: all of them) multiplies your share submissions by your difficulty. So yes, you'll submit 4 times more work at 128 diff, but it would be the same payment as somebody submitting a single piece of work at 512. Minimum difficulty has zero effect on your payout over any reasonable period of time.