Perhaps the banks concern isn't that you are buying bitcoin, but that you could be buying it and reselling it without meeting any local Anti-Money Laundering / Know Your Client requirements with each sale. In some jurisdictions it could be argued that if they suspected this were happening but didn't look into it that they may be an accessory to the crime. Similarly, a Credit Union might not want the risk of being associated with Bitcoin due to the media attention BTC has received in the past regarding links to crime.

Although you are just building your BTC savings, the bank don't know unless they ask. If they cause too many hassles then it might be worth considering 'voting with your feet' and banking elsewhere .