Something physical is not the mental approach you should be having towards cryptocurency.

What are your goals? Are you planning to buy hefty amounts, or just a few bucks' worth?
If you are looking to invest heavily, I would buy something like a TREZOR (a hardware wallet) which will keep your coins more secure. This WILL set you back over $100, so make sure it's worth it.
Otherwise, just use a lightweight wallet you trust. Electrum is a fairly good one IMHO.

Buying bitcoin can be as easy as dealing with a human being through a platform like localbitcoins, or as "complicated" as registering on an exchange that provides FIAT services, follow all KYC procedures, send a bank transfer, and buy coins from the exchange.

If you _do_ follow this path, do not store your coins on the exchange for the long-term. Always use a wallet that you can control, always encrypt your wallet, and always keep a physical handwritten copy of your recovery or private key. in a safe.

It's not rocket science, but you need to read around a bit, and try things out. practice makes perfect. don't practice on 15 grand at a time.