Does anyone know if this affects Armory wallets? if so, how do we upgrade safely?
Probably, Armory uses the bitcoin daemon.
Just upgrade the bitcoin daemon(packaged with bitcoin-qt/core.)
In reality, I've not seen any evidence to prove this vulnerability could effect a average end bitcoin user behind a router.
EDIT:
Modern routers do more than route network traffic.
They block incoming connections.
My understanding of the heartbleed venerability leads me to believe it would require a open port to request the memory dumps.
I felt I should probably elaborate on my above statement.