Objectively, I have always tried to follow this when linking any suspicious account-to-account activity:
If A does something that links it to B (in any way whatsoever) then this is an A-->B linkage. This means that B-/->A but A-->B. Any negative reputation associated with B can be transferrable to A but not vice versa.
Only when we have the same linkage originating from B can we safely say there is an A<-->B linkage.
This has been my protocol for the Known alts thread (as I have advocated in the past), though I know that some do not tend to dabble with this. Enacting this procedure would, however, prevent any foul play and false positives (though can miss the mark for some alts).
After all, if 5 accounts all used someone's addressfor example, theymosfor some giveaway, you wouldn't link theymos to those accounts, would you? On the other hand, if 5 accounts all used a scammer's address for a giveaway, then it becomes suspicious and you would trust those accounts less. However, you would still not link the scammer to those accounts.
There is more solid proof that they belong to the same person than what you are referring to.