If I trade with someone face to face for cash, the most they find out about me is the address I sent the bitcoin from. Since that bitcoin will have been mixed or otherwise anonymized, that tells them exactly nothing about me.

Centralized exchanges, on the other hand, track everything that you do. They know your name, address, phone number, email, occupation, income, SSN, tax details, and so on. They know your bank accounts and credit cards. Every deposit you make is analyzed via blockchain analysis companies. Every withdrawal you make is tracked to see where it goes. They use that information to link all your addresses and wallets together, and see how and when you spend your money. They link all of that to your real identity. They keep track of every trade you've made, every IP address you've used, every device you access their platform with. They use your IP addresses and device fingerprints to track your movements across the internet and other websites. They build an enormous profile on you, and then sell and share that information to a variety of third parties.

The two are not even comparable. To say a physical trade is worse for privacy than a centralized exchange is insanity.