Let's just use Gentoo without X11

If it's offline, it's safe. Stop being paranoid.
You joke but when you have lots of BTC it pays to be a little paranoid

One of the basics of computer security is to disable unnecessary services. For an offline Armory wallet, this means most of a traditional desktop install can go. I would like to see Armory more easily ported to other distros for this reason. X11 doesn't bother me so much (Armory is GUI only at the moment so you need it anyway) but stuff like the Ubuntu Amazon shopping lens, web browsers, and bluetooth software could be potential attack vectors. Sure you can remove all those from Ubuntu but I would prefer to start with a minimalistic Arch or Debian install with only X11, a bare Openbox for the WM, and the dependencies of Armory. Bonus points for blacklisting the kernel modules for the NIC and bluetooth cards so apps can't connect to anything even if they wanted to.