My second version is slightly improved, but I still have the bottleneck of list cloning. For instance, if I have a 1GB list and I use 4 cores for the search, it uses 4GB of RAM.
Write a separate section of the function for generating the list and call it in your main function.
but my problem is this
def worker( points....):
pass
points = set([1,2,3])
num_cores = 4
process = Process(target=worker, args=( points ...))
What happens is that when passing the 'points' list to each core, it gets cloned, resulting in 'num_core * points'.
