Post
Topic
Board Development & Technical Discussion
Merits 3 from 2 users
Re: The fastest way to download blockchain data in csv format
by
DougM
on 11/08/2020, 18:17:46 UTC
⭐ Merited by HCP (2) ,ETFbitcoin (1)
Hi,

I wonder how to download bitcoin data to csv files. I would like to get all blocks, transactions in csv files. What is the fastest way to achieve this?
I heard about bitcoin-etl and blocksci. Is it possible with both of them ?
It all depends on what you want to do with the data, but you might want to check out LoyceV's thread first to see if it answers your needs:

Without further ado:
The data
Directory:
loyce.club/blockdata/

All-in-one file:
|
|
|
Text file
blockdata.txt
blockdata.lastday.txt
|
|
|
Size (MB)
362
0.107
|
|
|
GNU zip compressed file
blockdata.txt.gz
blockdata.lastday.txt.gz
|
|
|
Size (MB)
129
0.042
|
|
|

This file is in CSV (Comma-Separated Values) format. It can be imported into a spreadsheet (this might take a while considering the size).
Sample:
The file blockdata.txt has the following columns with data for each block, the first line is a header:
Code:
id,hash,time,median_time,size,stripped_size,weight,version,version_hex,version_bits,merkle_root,nonce,bits,difficulty,chainwork,coinbase_data_hex,transaction_count,witness_count,input_count,output_count,input_total,input_total_usd,output_total,output_total_usd,fee_total,fee_total_usd,fee_per_kb,fee_per_kb_usd,fee_per_kwu,fee_per_kwu_usd,cdd_total,generation,generation_usd,reward,reward_usd,guessed_miner
0,000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f,2009-01-03 18:15:05,2009-01-03 18:15:05,285,285,1140,1,1,000000000000000000000000000001,4a5e1e4baab89f3a32518a88c31bc87f618f76673e2cc77ab2127b7afdeda33b,2083236893,486604799,1,0000000000000000000000000000000000000000000000000000000100010001,04ffff001d0104455468652054696d65732030332f4a616e2f32303039204368616e63656c6c6f72206f6e206272696e6b206f66207365636f6e64206261696c6f757420666f722062616e6b73,1,0,1,1,0,0,5000000000,0.5,0,0,0,0,0,0,0,5000000000,0.5,5000000000,0.5,Unknown
1,00000000839a8e6886ab5951d76f411475428afc90947ee320161bbf18eb6048,2009-01-09 02:54:25,2009-01-09 02:54:25,215,215,860,1,1,000000000000000000000000000001,0e3e2357e806b6cdb1f70b54c3a3a17b6714ee1f0e68bebb44a74b1efd512098,2573394689,486604799,1,0000000000000000000000000000000000000000000000000000000200020002,04ffff001d0104,1,0,1,1,0,0,5000000000,0.5,0,0,0,0,0,0,0,5000000000,0.5,5000000000,0.5,Unknown
2,000000006a625f06636b8bb6ac7b960a8d03705d1ace08b1a19da3fdcc99ddbd,2009-01-09 02:55:44,2009-01-09 02:54:25,215,215,860,1,1,000000000000000000000000000001,9b0fc92260312ce44e74ef369f5c66bbb85848f2eddd5a7a1cde251e54ccfdd5,1639830024,486604799,1,0000000000000000000000000000000000000000000000000000000300030003,04ffff001d010b,1,0,1,1,0,0,5000000000,0.5,0,0,0,0,0,0,0,5000000000,0.5,5000000000,0.5,Unknown
3,0000000082b5015589a3fdf2d4baff403e6f0be035a5d9742c1cae6295464449,2009-01-09 03:02:53,2009-01-09 02:55:44,215,215,860,1,1,000000000000000000000000000001,999e1c837c76a1b7fbb7e57baf87b309960f5ffefbf2a9b95dd890602272f644,1844305925,486604799,1,0000000000000000000000000000000000000000000000000000000400040004,04ffff001d010e,1,0,1,1,0,0,5000000000,0.5,0,0,0,0,0,0,0,5000000000,0.5,5000000000,0.5,Unknown
4,000000004ebadb55ee9096c9a2f8880e09da59c0d68b1c228da88e48844a1485,2009-01-09 03:16:28,2009-01-09 02:55:44,215,215,860,1,1,000000000000000000000000000001,df2b060fa2e5e9c8ed5eaf6a45c13753ec8c63282b2688322eba40cd98ea067a,2850094635,486604799,1,0000000000000000000000000000000000000000000000000000000500050005,04ffff001d011a,1,0,1,1,0,0,5000000000,0.5,0,0,0,0,0,0,0,5000000000,0.5,5000000000,0.5,Unknown
5,000000009b7262315dbf071787ad3656097b892abffd1f95a1a022f896f533fc,2009-01-09 03:23:48,2009-01-09 03:02:53,215,215,860,1,1,000000000000000000000000000001,63522845d294ee9b0188ae5cac91bf389a0c3723f084ca1025e7d9cdfe481ce1,2011431709,486604799,1,0000000000000000000000000000000000000000000000000000000600060006,04ffff001d0120,1,0,1,1,0,0,5000000000,0.5,0,0,0,0,0,0,0,5000000000,0.5,5000000000,0.5,Unknown
6,000000003031a0e73735690c5a1ff2a4be82553b2a12b776fbd3a215dc8f778d,2009-01-09 03:29:49,2009-01-09 03:02:53,215,215,860,1,1,000000000000000000000000000001,20251a76e64e920e58291a30d4b212939aae976baca40e70818ceaa596fb9d37,2538380312,486604799,1,0000000000000000000000000000000000000000000000000000000700070007,04ffff001d0123,1,0,1,1,0,0,5000000000,0.5,0,0,0,0,0,0,0,5000000000,0.5,5000000000,0.5,Unknown
7,0000000071966c2b1d065fd446b1e485b2c9d9594acd2007ccbd5441cfc89444,2009-01-09 03:39:29,2009-01-09 03:16:28,215,215,860,1,1,000000000000000000000000000001,8aa673bc752f2851fd645d6a0a92917e967083007d9c1684f9423b100540673f,2258412857,486604799,1,0000000000000000000000000000000000000000000000000000000800080008,04ffff001d012b,1,0,1,1,0,0,5000000000,0.5,0,0,0,0,0,0,0,5000000000,0.5,5000000000,0.5,Unknown
8,00000000408c48f847aa786c2268fc3e6ec2af68e8468a34a28c61b7f1de0dc6,2009-01-09 03:45:43,2009-01-09 03:16:28,215,215,860,1,1,000000000000000000000000000001,a6f7f1c0dad0f2eb6b13c4f33de664b1b0e9f22efad5994a6d5b6086d85e85e3,1716931356,486604799,1,0000000000000000000000000000000000000000000000000000000900090009,04ffff001d012c,1,0,1,1,0,0,5000000000,0.5,0,0,0,0,0,0,0,5000000000,0.5,5000000000,0.5,Unknown

Separate columns:
Instead of killing your spreadsheet, each column can be downloaded separately (the first line is a header). The column "id" (block number) is added to each file (except for id.txt) to detect errors. Make sure the rows are aligned when inserting the columns you need:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Text file
id.txt
hash.txt
time.txt
median_time.txt
size.txt
stripped_size.txt
weight.txt
version.txt
version_hex.txt
version_bits.txt
merkle_root.txt
nonce.txt
bits.txt
difficulty.txt
chainwork.txt
coinbase_data_hex.txt
transaction_count.txt
witness_count.txt
input_count.txt
output_count.txt
input_total.txt
input_total_usd.txt
output_total.txt
output_total_usd.txt
fee_total.txt
fee_total_usd.txt
fee_per_kb.txt
fee_per_kb_usd.txt
fee_per_kwu.txt
fee_per_kwu_usd.txt
cdd_total.txt
generation.txt
generation_usd.txt
reward.txt
reward_usd.txt
guessed_miner.txt
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Size (MB)
4.1
39
12
12
3.8
3.7
4.1
2.9
2.6
19
39
6.4
6.0
9.1
39
61
2.3
1.5
2.4
2.4
6.9
5.2
7.7
5.4
4.4
4.2
3.9
3.6
4.0
3.5
8.2
6.6
4.3
6.6
5.4
5.0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
GNU zip compressed file
id.txt.gz
hash.txt.gz
time.txt.gz
median_time.txt.gz
size.txt.gz
stripped_size.txt.gz
weight.txt.gz
version.txt.gz
version_hex.txt.gz
version_bits.txt.gz
merkle_root.txt.gz
nonce.txt.gz
bits.txt.gz
difficulty.txt.gz
chainwork.txt.gz
coinbase_data_hex.txt.gz
transaction_count.txt.gz
witness_count.txt.gz
input_count.txt.gz
output_count.txt.gz
input_total.txt.gz
input_total_usd.txt.gz
output_total.txt.gz
output_total_usd.txt.gz
fee_total.txt.gz
fee_total_usd.txt.gz
fee_per_kb.txt.gz
fee_per_kb_usd.txt.gz
fee_per_kwu.txt.gz
fee_per_kwu_usd.txt.gz
cdd_total.txt.gz
generation.txt.gz
generation_usd.txt.gz
reward.txt.gz
reward_usd.txt.gz
guessed_miner.txt.gz
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
File sizes on May 6, 2020. I won't update the size as they grow each day.

Background
In Bitcoin's Empty Blocks Analaysis, mikeywith asked if I can help scrape data for a 10 year analysis. Tranthidung posted a link to Blockchair Database Dumps, which offers compressed block summaries with daily updates.

Updates
All files will be updated daily.

Credits
Blockchair Database Dumps has a staggering amount of data, easily accessible in daily files (at 10 kB/s). All data presented in this topic comes from Blockchair.
I've converted these files to make it easier to access all data at once.

No spam please.
Self-moderated against spam. Discussion and questions are welcome.



Related topics
Bitcoin block data available in CSV format
List of all Bitcoin addresses with a balance
List of all Bitcoin addresses ever used