There isn't enough data storage on earth, maybe not in the entire universe, to remember all primes, so finding new ones surely has to involve forgetting old ones at some point.

Primecoin though isn't looking for primes, it is looking some some fancy pattern/sequence of primes, I dunno though whether they too are so numerous we won't be able to remember them all.

Though if we only need remember on the blockchain those we found for the blockchain that isn't a problem for us; and maybe only bothering to remember some rare sequences is going to let science actually remember what is found instead of trying to find stuff it is only going to have to forget at some point when it runs out of memory/storage...

-MarkM-