I know there are a lot of them and maybe we won't use them all
It seems that you don't know how addresses are generated.
It's not that there's a big list of addresses and every time you generate a new address, you pick one from the list.
If you worry about generating an address that has already been generated by someone else, note that in theory it can happen at any time. In practice, it will never happen.