If every transaction in the blockchain is broadly known to be the result of a swap, it will be harder for a potential observer to find out who owns what, and who makes transactions with whom. If a robber is not sure you really have those coins, he is unlikely to make you a target.
That would require a serious attitude change of the entire Bitcoin userbase and I am sure that is never going to happen. But even if it does at one point in the future, I don't see robbers going that deep into research to consider if their potential victim has or doesn't have the coins that their history shows they should possess.