Regarding hackers, they might reinstall those drivers if they manage to get physical access or if you use any infected devices.
Your disks should also be fully encrypted and password-protected to prevent anyone recovering any data from them in case your machine gets stolen. Remote access shouldn't be a worry in a properly airgapped computer. If it is, then it's not properly airgapped, and you are just adding more risk to the way you store your keys.
There may be hardware backdoors, OS zero days, perhaps the BIOS is compromised. Nothing is 100% secure, but if you migrate to an air-gapped environment, you definitely have less things to account for.
Even if there is a vulnerability, there is no way to steal and send that data to wherever it's supposed to go without networking. If you connect USB devices to it to transfer PSBTs, I doubt a malware can be that good to transfer your data to the USB device and then wait for you to connect it to an online machine to transfer the information over the internet.