But Cannabis different. That's illegal on a federal level, sex or porn isn't so I can't see any justification why they can't deal with adult sites. Adult sites must have bank accounts too and lets not forget its a billion dollar industry so some banks must be working with them.
Both Cannabis (at least in Colorado) and porn are not strictly illegal,
de jure. However, the banks treat them as "risky sectors to do business", and in 99% of the cases, they will refuse to set up savings accounts for them. Therefore we can say that the banks are treating them as
de facto illegal types of business, along with other sectors such as merchants dealing with arms and ammunition.