Very specific to the implementation of your client. If new addresses are generated from a common seed then you need to make sure the xpub never leaks. How does your client looks up the utxos? What if your computer is compromised? Does the software leave any trace on your computer? Unencrypted logs etc? Only one things needs to go wrong. Heck, even if the implementation is perfect, a simple malware attack could link your addresses.