DarkSend question:

After a DarkSend, the change is sent back to a hidden change address to which I own the private key, but no one knows it's mine. But what if I later had to send an amount large enough that my wallet is forced to source it from multiple addresses including this change address? This could then reveal that the original sender address and the change address are connected. With the change amount known, the sent amount is also known and the recipient can be guessed more easily. Is this correct?