We have evidence in this thread of users not wanting to accept Segwit transactions. That's users not miners, that's a lack of fungibility, and that's an issue.
I didn't know that. If you gave me a legacy BTC address (if we had some sort of deal or transaction), I'd send you some BTC that came from a native segwit address. You'd still get your BTC, and you'll likely get it confirmed in a block faster too. Why would you not accept that? Would you return it?