Quote from: LoyceV on Today at 09:28:23 AM

Quote from: PrimeNumber7 on Today at 04:30:42 AM

Quote from: LoyceV on June 03, 2021, 02:55:10 PM

Update: I got you http://107.191.98.18/addresses_sorted.txt.gz ! It's 19 GB. Please let me know when I can nuke the VPS again.

"Your download will take ~4 hours to complete"

I get this (in England):

Code:

- 0%[ ] 139.10M 33.9MB/s eta 10m 15s

The 4 hour quote appears to be the result of my crappy WiFi connection on my back porch. I was able to reproduce a ~10 minute download estimate via a datacenter. I have been able to transfer ~a half terabyte worth of videos stored in a storage bucket in seconds.

Quote from: LoyceV on Today at 09:28:23 AM

Quote from: PN7

IMO, you should upload the file to a GCS/AWS/Azure/Oracle/etc storage bucket, set the permissions to "anyone can access" but set the object so that the "requestor pays" for downloads. This will result in you paying under a dollar per month in storage costs, but anyone who accesses your file will pay a few dollars to get your data in seconds.

There's 2 problems with that: I don't want to use a creditcard, and I don't want anyone who downloads it to require a creditcard. If I need to charge a few dollars per download, I'd rather set it up myself to accept Bitcoin payments.

That is a reasonable desire, however it is something that is more difficult as you are making big data available to the public. Service providers have limited network infrastructure, and need to pay for data sent to the internet, regardless of if they have data caps, or charge you for egress/outgoing data. If you are freely sharing a 10 or 20 GB file(s) using a service provider that does not charge per data transferred, you will eventually get kicked off from that service provider.

Another point is that many people look at bitcoin-related data today. The fact that someone is looking at blockchain data is not the privacy leak that it might have been 10 years ago.

Quote from: LoyceV on Today at 09:28:23 AM

Quote from: PN7

Maintainng a multigigabyte file that is accessible to the public for free, that can be accessed unlimited times is really not feasible.

My other project (List of all Bitcoin addresses with a balance) is closing in on it's 2 TB montly bandwidth limit. I'd hate to have to setup a payment system, especially since this is basically just mirroring data from Blockchair.com.

There is a reason why blockchair throttles downloads, and why they charge as much as they do for an API key.