It's not quite that bad. If one address is compromised, and your encrypted wallet file is also compromised (even if the password remains unknown), then some of the rest would be as well (addresses, including invisible change addresses, that are created after the one that's compromised).

It's up to you to decide how risky you think this is...

If you have a good reason to import some addresses to an online wallet, you could also create a new Armory wallet and try to keep most of your bitcoin in the new wallet, and treat the convenience wallet as less safe.