You can instead just delete the wallet file, since that's the place where they keys are stored. Deleting and redownloading the software won't make you more secure, it might even add a small risk, because each time you download a wallet you might accidentally download a fake wallet.
The first is as harizen points out - you will be unable to verify your receiving addresses. If you are subjected to clipboard or other malware, the addresses you saved to receive to may be edited at some point between storage and entering them to a website/handing them over to someone who wants to pay you. You have no reference point to double check they are correct. This could be mitigated by printing out your addresses and double checking them against your wallet prior to deleting it, and then using your print out as a reference for all future address use.
Green wallet has a watch-only mode, which solves this problem.