Re: List of all Bitcoin addresses ever used

Quote from: LoyceV on August 01, 2020, 09:05:46 AM

Updates
Sorting a list that doesn't fit in the server's RAM is very slow. Therefore I only update unique_addresses.txt.gz twice a month (on the 6^th and 21^st). Check the file date here to see how old it is. If an update fails, please post here.
In between updates, I create daily updates: alladdresses.loyce.club:20319/daily_updates/. These txt-files contain unique addresses (for that day) in order of appearance.
Due to limitations in disk space, I don't do automatic updates for addresses.txt.gz. It's complete until blockchair_bitcoin_outputs_20200719.tsv.gz.

This is a wonderful initiative! A comment: Sorting a very large list with little RAM is not necessarily a problem! Try:

Code:

mkdir tmp
cat unsorted.txt | sort -u -S 65% -T tmp > sorted.txt
rm -r tmp

-S will tell your machine to use at most 65% CPU; this is some sort of optimum, according to my experience
-T puts temporary files in a directory (here named "tmp") and not in RAM; if you have an SSD, the speed isn't too shabby

I have sorted huge lists (>80 GB) on budget laptops using these two arguments. Worth a show! If you want better hosting, PM me.