Welcome to the forum @Hopelesser

You don't have to download "their wallet" I have never seen an exchange website having their own wallet and to be honest I surely wouldn't trust the whole website.

1) You have to download your own wallet, such as "Electrum"
2) Then you go to the website and buy some bitcoins or any other altcoins, as you wich
3) Then you move the coins from the exchange to your wallet

If this is the path you want to follow, consider the transactions fees.