Id prefer just to write down my public and private address and thats it if possible.
Yes. Just not easy. I'll explain further down.
I guess my first question is what really is a wallet? Is it the private and public address combination?
A wallet contains 1 or more public/private bitcoin keys.
If I keep my bitcoins in my wallet and do not use exchanges what program/website do I go to when I need to send money to someone? Or how do you check your balance?
Ok. Let me try to explain.
A bitcoin private key is just a number. In fact it's 256bits. Usually a random number. Go to brainwallet.org and click the random button (RHS) you will see a 256bit random number in hex. You will also see the bitcoin key that goes with it. You could write these both down if you wanted to.
If you then want to send any BTC on that address, you can load up brainwallet with the private key, and it has menu option Transactions that will allow you to make a payment.
Optionally....
Just open a
https://www.strongcoin.com account. It's easier.