Sure they could. An attacker could use blockchain analysis to reasonably conclude a lower bound of how much coin a person is holding. An attacker could use a set of known facts to conclude a person is holding a large amount of coin.
If your coins are spread around in different wallets and these addresses are not public and can't be connected to your real name it is safe to assume that you would be safe. How would someone discover that a certain number of addresses belong to John Smith from Denver, Colorado?