Now I'm afraid to start Bitcoin Core if it will start moving funds around. I have all keys backed up so that's not a problem.

I already have a follow up question:
What's the best way to move funds from a specific address that I have in Bitcoin Core?

EDIT: I just do "dumpprivkey YourBitCoinPublicAddress" never mind my question!