There wouldn't be any point exchanging only $1 for Bitcoin.  You won't be able to send it anywhere due to high transaction fees on the Bitcoin network right now, and most exchanges will not take a deposit that low.

Depending on your country, your best bet will either be to sign up for an exchange like Coinbase, for which you will need to spend a $25 minimum deposit, or with P2P trading like on LocalBitcoins, where you can trade with bank transfer or cash.

It is alright to avoid using a credit card but in particular you shouldn't use PayPal because transactions with Bitcoin are against their ToS and there will be very high fees dealing with PayPal because of the high risk of people reversing their transactions.