I'm not sure how it works in the US. In Israel, where i'm from, banks are regulated and can't refuse business without legal ground. About a month ago the regulator "released the banks from their duty to provide service for gambling sites" - http://www.yogonet.com/english/2012/01/11/israel-banks-can-refuse-to-handle-transactions-for-suspicious-websites
And take my word for it, Israel is not the most liberal country when it comes to human rights...