Not quite. The 2⁹⁶ private keys in question will produce 2⁹⁶ different public keys. It is the public keys which undergo RIPEMD-160 to produce the same output which will encode to the same address.

So different private keys, different public keys, same address.