LoyceV post the code!
This works (on Linux) to display addresses that exist in both lists:
# Only once:
cat blockchair_bitcoin_addresses_latest.tsv | cut -f 1 | sort | uniq > alladdresseswithbalance.txt
# Option 1:
comm -12 alladdresseswithbalance.txt <(cat mylist.txt | sort | uniq) # fastest
# Option 2:
cat alladdresseswithbalance.txt <(cat mylist.txt | sort | uniq) | sort | uniq -d # 5 times slower
Option 1 takes 6 seconds on my old laptop, checking 300,000 addresses against 30 million addresses with balance.
For various reasons, I'm more interested in transaction history - e.g. "even if the balance is 0, has this public address ever occured on the blockchain?" - and for that you need more input than only Blockchair's list of public addresses that have a positive balance "now".
Blockchair also has
all outputs, but at 10 kB/s it's going to take a while. If you can get all addresses ever used from your own version of Bitcoin Core, you're good to go.
Yup. Downloading Bitcoin Core, letting it sync and run a blockparser will do the job. It's not fast and easy; in fact, most if not all block parsers I've tried require almost insane hardware and a decent amount of patience. And some bash skills (such as yours) to produce lists.
Provided you have a full node up and running (i.e. a folder containing .blk files), which block parser would you recommend today? I updated my own list of "all used public addresses ever" about a year ago, which gave me a list of around 500 million addresses, and it took some blood, sweat and tears (and curse words). Also, I was never 't able to extract bech32 addresses (bc1...). Since I haven't looked into recent development in the field, I'm interested in knowing which one on github one should go for today. Many thanks.