One drawback I see is that if the house burns down I would lose everything I manage with one HW, but well, that risk I also had now, having 100% of the management of my holdings in one site.
That can be addressed too. Instead of saving your backup seed(s) only onto paper, if the amount of funds worth it you can also save onto steel - from CryptoSteel Capsule to Willi's steel plate or, if you want it cheaper or in a way that doesn't draw attention, then onto washers.