Re: List of all Bitcoin addresses ever used

Quote from: LoyceV on August 01, 2020, 09:05:46 AM

Background
To follow up on List of all Bitcoin addresses with a balance and this post, I made a list of all Bitcoin addresses that have ever been used.

The data
See alladdresses.loyce.club (new location)
I now have the resources (RAM, CPU power and disk space) and code to show unique addresses in their original order. Each address is only shown once. I have 2 large files:

1. All Bitcoin addresses ever used, in chronological order, without duplicates.
Sample: addresses_in_order_of_first_appearance.txt.gz: (Warning: 23 GB):

Code:

1A1zP1eP5QGefi2DMPTfTL5SLmv7DivfNa
12c6DSiU4Rq3P4ZxziKxzrL5LmMBrzjrJX
1HLoD9E4SDFFPDiYfNYnkBLQ85Y51J3Zb1
.......
3GFfFQAFgXKiA1qqUK6rqBpEpG4vZDos6t
3Mbtv47gZ2eN6Fy7owpgHHwSLYHS42P56P
38JyF2RQknBUMETyRT2yGndDJFYSp6hJNg

2. All Bitcoin addresses ever used, sorted by address, without duplicates.
Sample: addresses_sorted.txt.gz: (Warning: 20 GB):

Code:

1111111111111111111114oLvT2
111111111111111111112BEH2ro
111111111111111111112xT3273
.......
s-ffd80dee5966fb23c1a483b28f6bfcbc
s-fff5d0faa9628c188e97661f0e185fce
s-ffff291613d413b4ac128df96a462294

Updates
Sorting a list that doesn't fit in the server's RAM is slow. Therefore ~~I only update both large files (addresses_sorted.txt.gz and addresses_in_order_of_first_appearance.txt.gz) twice a month (on the 6^th and 21^st, updates take more than a day).~~ Check the file date here to see how old it is. If an update fails, please post here.
In between updates, I create daily updates: alladdresses.loyce.club/daily_updates/. These txt-files contain unique addresses (for that day) in order of appearance.
I won't keep older snapshots.

Bandwidth
This server is allowed 5 15 TB bandwidth per month. If it runs out, it probably gets suspended Tongue

I haven't tried that yet.

Credits
Blockchair Database Dumps has a staggering amount of data, easily accessible (at 10 kB/s (or recently 100 kB/s)) with daily updates. All data presented in this topic comes from Blockchair.

No spam please.
Self-moderated against spam. Discussion and questions are welcome.

Q&A

Quote from: MrFreeDragon on August 17, 2020, 05:56:46 PM

Can you please clarify, what is the type of these d- and s- addresses?

This is how Blockchair.com shows OP_RETURN. From the main page the search field doesn't show them, but you can replace a Bitcoin address in the URL to find them: https://blockchair.com/bitcoin/address/d-d0d953f2e7043342540a1407243e49fe.

Tips and tricks
Some suggestions for Linux/VPS users:

Code:

wget http://alladdresses.loyce.club/addresses_sorted.txt.gz -O - | gunzip > addresses_sorted.txt

This doesn't save the .gz but extracts it while downloading.

Code:

comm -12 <(sort list.txt) addresses_sorted.txt

This outputs all Bitcoin addresses from "list.txt" that have ever been funded.

Code:

comm -12 <(sort list.txt) addresses_sorted.txt > output.txt

This does the same, but writes to output.txt instead of console.
This search is fast, even with millions of addresses in list.txt, it's mainly limited by how fast your computer can read from disk.

Interesting statistics!! Thanks for sharing an information!!