It looks like the other's have answered your question on the .exe portion, but I would also recommend that you backup your wallet.dat file from the installation directory. Generally that is the file that contains your keys. I would keep it either on a flash drive or a second copy in another form of safe storage. Just in case. We threw out a computer in 2010 that had bitcoin worth a few hundred thousand dollars nowadays. Hindsight is 20/20.