Quote from: crofrihosl on September 24, 2019, 05:30:52 PM

i use windows so yes it has to be python

You can use bash utilities on Windows too. Or boot your computer from an Ubuntu LIVE DVD just to do this.

Quote

yes it is very very slow for files above 1,000,000 lines

when i want to compare a files with +10,000,000 lines to other 2,000,000 lines (i had to cancel and close the script Undecided

)

It took me a while to create 2 test-files with 50,000 Bitcoin addresses, so I just copied the same addresses to the same files to make it 10 million and 2 million lines per file.
The comm-code above took 9 seconds to find all matches (and my PC is not very fast). I strongly suggest to use the proper tools for the job

Update: it used 1.5 GB RAM to do this. If you have much more data to compare, it might reduce memory load if you sort the files first.