I recently created a bunch of paper wallets by downloading software and creating some addresses while I was offline. That said, do I now need to activate these addresses somehow? (I guess I'm kind of wondering how the exchanges will know that my addresses actually exist if I created them while I was offline).
There is no activation. Addresses are not registered or activated or anything. They are just used.
What are you trying to do? I don't know why an exchange would care if your paper wallets have bitcoins or not. If you want to send bitcoins to a paper wallet, you simply enter the wallet's address where the site tells you to.