The first you should do is checking the wallet or miner file before you install it on your PC , make some research about it and the project.
I'll have to look in to how to verify for myself if a new wallet is safe. I assume you get them from github.
I haven't taken the time yet to see what one would look like, if the wallets have code that is readable, or if it's all in machine code and you need a tool to convert it to readable form to look for anything malicious.
I do still want to get in to GPU mining too and I think that would be a lot of fun, though now, it's a little more than I can tackle atm. At least now I know to be more aware of security and not just go willy nilly with installing new wallets.
Regardless, I would definitely use a VM and make sure the network won't be compromised.
Its never too late for get into mining when you have low electrical costs or even better electricity for free.
I still have loftier ideas and haven't given up on mining. I do really hope to have a decent mining farm someday, like after I have a solar system installed. It might take a while.