The old QT wallet is simple the executable. Deleting it will uninstall it. However there are files stored in the folder c:\users\(your user name)\appdata\roaming\bitbay

However just remember in that folder is wallet.dat which contains your account so make sure to back that up or move funds out of that wallet before you intend on deleting that folder.