See, if it were dollars, I could just reach into your bank account or your mattress, or wherever you keep them, and take them. Can't do that with bitcoins.
That's just out of practicality.
Anyone who possesses your private key has your coins.
So its more like bitcoin gives you a very strong box for holding your value and gives you a key.
This key can be taken from you and you could lose it.