I personally don't use VPN to stay anonymous simply because I don't trust that VPN company won't keep logs of who am I and of my activity.
Your ISP definitely keeps logs of everything you do, and they are linked to your real name and address. At least with good VPNs you can sign up anonymously and pay in anonymized bitcoin.
-snip-
I used VPNs and Tor a long time before I used bitcoin. I am much more concerned about blanket surveillance by governments and large corporations than I am about people finding out I own bitcoin. If speed is an issue then I would shop around - you should be able to find one that only gives you a 2-3% speed reduction.