some more thoughts...

You should download from them your "private keys" if you want a backup (just make sure you don't have a virus, etc, so they arent stolen)

You could then import that to Bitcoin (the client .exe program) or any other site.

The private key is what really matters, it's what lets you spend money from an address.
I assume blockchain.info must have it, otherwise they couldn't do anything with your coins.
But I think they don't store it all  - only part, that when combined with your Blockchain.info password, gives them the entire private key. That's my guess

FYI A wallet is made from several addresses , each has its own private key

example, my address

13oNwed89qeuUWxredNj465D4sEdMuJ9NQ

has a private key that , if I import it into bitcoin.exe running on a nother computer, lets me spend from it