In my opinion, Ordinals is harmless as long as there is no significant number of people who take it seriously. Otherwise, the fungibility of Bitcoin could become an issue.
If by "fungibility" you mean the property that each unit is interchangeable with another, then I would disagree. Unless you believe that "taint" has compromised fungibility, I don't think it's accurate to say so. While "taint" has certainly affected the space by misleading people into believing that some coins might be "worth less", reality proves otherwise. You can trade any bitcoin for any other bitcoin, making it fungible. The same principle applies if more people start taking Ordinals seriously; as long as my coins can be exchanged for equal value, it doesn't matter what others do.