Is there an actual way to permanently delete such files from the HDD, forever?
If the HDD is working, there are various software around that can overwrite existing files with garbage before removing them (wipe) and there are programs that can do similar stuff with the current free space on your HDD.
If you're under Windows I recommend
SysInternals SDelete for the job, since SysInternals were bought by Microsoft many years ago, hence it should be safe.
I'd rather not have someone recovering my wallets this way.
If you want to super-sure and super-safe, why don't you just create brand new wallets and move your coins there?