Quote from: etotheipi on September 01, 2013, 06:27:39 PM

Quote from: pankkake on September 01, 2013, 05:42:03 PM

I'm fine with the storage requirements, especially if it means being able to watch any address easily. Storage is cheap and easy to add.

Yup. And at least you get something useful out of all that extra storage space: you can get the entire history for any address (including multisig TxOuts) nearly instantaneously. This will make armoryd.py quite a bit more powerful than bitcoind, since it will ultimately implement everything bitcoind does plus all the arbitrary addr/script lookups. It also means that you can import addresses and wallets without having to rescan.

And with that, it may make sense to implement a basic python shell into the expert interface, that allows users to issue commands/requests directly from a "debug" window. For instance, you'd run "Server Version" GUI (not armoryd.py), and open the "Armory CLI" window, and type in "getAddressHistory('')", "importAddress('', '')", "findAllMultisig('')", etc. I could make the same interface available through RPC, but that will take considerably longer because of how security-sensitive that is.

This super-node version also enables a very specific functionality that is not otherwise possible: encrypted watching-only wallets. Without a full address index, and without rescanning the blockchain every load, there's no way to hide the addresses in your watching-only wallet. My new wallet design (which was side-tracked by this RAM-reduction effort) actually has a way to encrypt the public keys in a WO wallet, but it's not very useful if your database is storing only the transactions related to your wallets. There's things you could do to improve the situation, but it won't be perfect unless you do a rescan every time you unlock your wallet and hold everything in RAM. Or save the whole tx history as this super-node version is doing.

So if you want absolute security and privacy, you can run this "server version" which will use a lot of HDD space, but it will run smoothly once the DB is built and you can hide the public keys in your wallet. A lot of people have requested this functionality, because of concerns that a stolen laptop could lead to worst things than losing money. For instance: the thief may not get your $100k worth of Bitcoin since he only has your WO wallet, but he can see that you have $100k worth of Bitcoin and can probably find your address somewhere else on the computer. Many of my more paranoid/rich users would like to avoid that.

Profiling Update: my initial DB build is up to 221,000 blocks after 1.5 hours, and using 295 MB RAM (15 MB more than at 180k blocks). So it looks like it may take about 3-4 hours to rebuild the whole DB and about 300 MB RAM. Those are some solid, usable numbers!

Not only that, but it's very likely that I can build the initial database while Bitcoin-Qt is downloading the initial transaction history, meaning that Armory's DB building can be "hidden" and the whole new-user-startup may not be any worse than running vanilla Bitcoin-Qt. I will not complicate it with that parallelization right now, but those numbers are very encouraging for a future upgrade.

Okay, enough rambling, more coding

...

Your progress sounds damn cool!!!!! Better honestly than i was expecting!! Ive been fretting on other threads about this in the past, but the world urgently needs a solution that allows local blockchain browsing a-la blockchain.info. So if you can also deliver an engine that efficiently does this without a horrible rescan every time, then Armory "server" will have a killer app that others will lust for! :-) This is totally above and beyond the existing core coolness from your current feature set!

Woo hoo, ill tip you again fat style when the day comes that i can do an arbitrary blockchain query without waiting ages for a chain scan, and without telling everyone who is spying on web traffic the addresses i'm interested in by visiting blockchain.info. They have a great service but frankly it's dangerous. A query there is enough to publicly link your real world identity to those addresses that you are interested in. Tor has its limits in terms of performance and market penetration.

No reason my isp or random law agencies deserve this info, its enough that the spies of the world could filter and log my transactions if they wanted to. They don't deserve to also know all the other addresses i might peek into.