Do you know if there is a way to use a passphrase with this HD seed like we can do with BIP39 memonic seeds? With a Bitcoin Core feature or an external tool.
So far, there's no BIP to encrypt HD/BIP 32 seed with password/passphrase. The other option is use common encryption algorithm (such as AES-256) through offline/open-source tool such as
https://cryptotools.net/aes.
Take note that USB flash drives have 10 years lifespan if you want to hold them for a long term make sure to make multiple copies to other storage or devices like CD/DVD discs these two have a long lifespan of around 30 to 50yrs.
CD/DVD only have long lifespan if you buy good ones and store it properly. You might as well as buy specific storage which specially designed for long term archive, such as M-Disc or SanDisk Memory Vault.