Why exactly shouldn't you do it? Possible problems with escrow if needed?
Because you can't prove that you are the owner of the sending address. You don't hold the private key, can't sign messages with it etc. In short, don't do it. Ever.
That's true, but isn't the higher risk here that the exchange is disappearing with your coins? If the coins are successfully at their destination, does it matter what the sender address was?
Or am i missing something vital here?
Not having access to your private key means you literally don't own the coin. The exchange does. Look at ETC, 1000s of people lost $1000s because exchanges simply refused to split the coins and they never received them. Nothing they could do.