You're not getting it. The concept of a non-transparent blockchain precludes there being "bad" outputs. There is, in general, no-deanonymiziing (certainly no assurance of it) and just outputs (coins) and transactions.
That is my point. With mixing on the block chain, there is no way to rollback just the double-spends if they get extensively mixed in with the legitimate transactions. Thus I do have to fear that when I spend, it can be unwound and that is me doing a double-spend, because my recipient has lost his funds. What if I've died, moved on, lost or discarded my private key, etc.. I can't reissue the transaction.
And on chain mixing could in theory much more amplify this potential Gordian knot.
And please don't equate with waiting for 6 confirmations as you did upthread. Different risk category when we are talking about an extended duration fork attack.