If every transaction in the blockchain is broadly known to be the result of a swap, it will be harder for a potential observer to find out who owns what, and who makes transactions with whom. If a robber is not sure you really have those coins, he is unlikely to make you a target.
That would require a serious attitude change of the entire Bitcoin userbase and I am sure that is never going to happen.
The protocol (PayJoin[1]) already exist though. If more merchant use it as default option and more wallet support it[2], i expect some users will use it and some of them might not even realize their transaction use PayJoin.
But even if it does at one point in the future, I don't see robbers going that deep into research to consider if their potential victim has or doesn't have the coins that their history shows they should possess.
I agree, it's more likely the robber use another means to obtain reliable information or simply seek more vulnerable target.
[1]
https://github.com/bitcoin/bips/blob/master/bip-0078.mediawiki[2]
https://en.bitcoin.it/wiki/PayJoin_adoption#Software_Wallets