There's no way those addresses can be part of another wallet, they either get used by you or they'll never be used.
There's nothing stopping those addresses being part of another wallet. Two different seeds could absolutely generate the same address somewhere in their derivation paths, and indeed, since the number of all possible derivation paths is
many orders of magnitude higher than the number of possible private keys, then it's certain that this is the case. But of course this will never happen simply because of the sheer size of the numbers we are dealing with.
I feel OP (wrongly) thinks that there are limited number of addresses.
But there
are a limited number of addresses. It's just that we will never reach that limit.