I had a question in mind how could we mitigate this if I have only access to a single physical pc ?
I don't think if there is any way to mitigate that without using an additional device. Did you consider buying a hardware wallet? You wouldn't have to bother with a separate computer or VMs if you bought one.
edit: if you could as well please tell I what is wrong in my setup beside the threat model you shared.
Besides that threat, it is a little difficult to maintain because you are using Bitcoin Core. You wouldn't have to worry about keeping your client up-to-date if you used Electrum, for example.