You can buy it with paypal through localbitcoins.
If you don't want to verify your ID, it's probably because you don't want your name attached to the transaction... so if you make a transaction through PayPal, well your name will be on that account!
Assuming that exchanges that verfiy your ID share the info with tax agencies (not yet, but inevitably), the only way you can make such a transaction anonymously is in cash. And that's why cash transactions are much more expensive. You can do a local bank transfer on localbitcoins for as low as 2% (I even got -1% once), but cash will be closer to 6%, even 10% depending on where you live. And assuming that someone is there.