This! People tend to forget, especially in the never-ending discussions about the coffee cup buy with BTC that for the merchant to actually use those coins it would mean he would have to pay at least one tx also, even if it will be a consolidating one. On top of what you mentioned there is also the human resources cost, with a lot of users there will be a lot of newbies who have problems with depositing and withdrawing, you need extra staff for support, you need also to pay the guys that actually check and validate the internal movement of all those coins, at the end of the costs can easily go way beyond that small tx fee.   

Of course, right now it's no excuse for some of the fees exchanges are using that have also not been adjusted with the price, like Binance's 0.0005 fee that really needs another 50% adjustment , 20$ is close to robbery, they make more from that than from actually trading fees.