Simple answer.
1. Yes each wallet must be downloaded unless it has an online wallet. Never use an exchange as a wallet.
2. You will lose your coins if wallet.dat is deleted. Find wallet.dat here :
First search %appdata% hit enter.
Look for the coins name and open folder. A wallet.dat file will be there. Back it up !!
To restore your wallet or to change to another one simply replace the wallet.dat file.
3. Use this forum.
So I decided to take the dive and start backing up my wallets, but wanted to double check on a few things which tie specifically into the post above:
1. Do I just need to backup the "wallet.dat" file or the whole folder that contains the wallet.dat file in my - %USERPROFILE%\AppData\Roaming\"coin name here"
If you just want to recover your wallet, then wallet.dat file is enough. But you would have to download entire blockchain again to spend it from your client.
2. If I change the name of my wallet.dat will it corrupt the file. For example, say I have like 5 different coins and 5 different wallet.dat files I wouldn't know what is what. (unless of course I make different folders that labeled the coin...yes I know it's a simple solution, but just wondering) So if I changed them to say bitcoinwallet.bat & ethwallet.bat etc. etc. would those files still work? OR could I do that, but have to just eventually change it to wallet.dat when I want to use it again?
AFAIK, renaming wallet.dat won't corrupt your file. You will have to change extension to .dat to recover your wallet.
3. If I delete a wallet, then reinstall a wallet, do I just need to go into this folder %USERPROFILE%\AppData\Roaming\"coin name here" and just replace/overwrite the "wallet.dat" file once the new wallet is done syncing with the network?
You can replace wallet.dat (default) file with your wallet file anytime. Don't forget to close your client when doing it. Though you can do it anytime,, I suggest you to do it before syncing, so you don't need to rescan to check transactions.
I figured I'd start with a wallet that has a small amount in it so in case I screwed something up I wouldn't lose much.
You can use testnet instead.
https://en.bitcoin.it/wiki/Testnet