I indeed generated 20 million adresses for no reason which resulted in 2 GB of data...
The extracted keys or the addresses alone are roughly 750 MB each. the rest is "overhead" by the generator.
I think I will create a query on bitcoin-wall.com/check/ querying each address :/
Proly still gonna be slow as heck but I could split it on multiple threads...
Won't have any hits anyway, so you guys are probably right that I shouldn't import them if I just want to check if they have a balance..
But I was more interested in owning them "forever" in the first place.
Are there any solutions (maybe bitcoind?) that can handle a larger amount of adresses than the desktop solution?
It's unrealistic to expect that a service would allow you to do 20 million requests in any reasonable rate. You should run your own full node and check the addresses yourself. Unfortunately there seem to be
no easy way to do it , which means you'll have to write your own code that works with raw blocks or UTXO set.
Or maybe you should stick with your method but modify it to periodically flush these empty addresses to free memory and save resources - e.g. delete all empty addresses every 1000 addresses.