but still note down your private keys and keep them in a safe
How exactly? I'm not sure how I would find them.
To do this in Bitcoin Core, you'd need to open the debug console, unlock your wallet with the walletpassphrase command, then use the dumpprivkey command on every address in your wallet that contains coins. This is a laborious process, and it's much easier to simply back up your wallet.dat file (hopefully you encrypted it with a password that you won't forget) which contains the deterministic key for your wallet, meaning if you back up that file, you won't need to replace it with a new version every time you receive coins.