This is the main issue, allowing a third party make unilateral decisions on how you spend your assets is not advisable. If censoring addresses becomes a trend among exchanges it could lead to then asking to verify and whitelist recipients of funds from their website even if those users do not use their platform.
It could also give their customers an impression of faux safety, so any address they're allowed to send to is considered legitimate.