Thanks for the explanation on that attack vector, very helpful.
All public keys and attached addresses that the wallet can have + any comments you left in there.
At the risk of sounding really dumb, does the fact that the hacker can match the public key to the addresses in the wallet make a brute force attack on existing address balances more likely?
No. The point of a public key is to be public and shared with others without revealing the private key. The address is derived from the public key so it too can be shared. If it were possible to derive the private key from the public key, then the entirety of Bitcoin would be broken and there would be a massive problem not just with Armory but with Bitcoin itself.
All that would happen if someone got your watching only wallet is that he would be able to see every single transaction you make. At worst it is just a privacy leak for you.