Hello,

I've been playing around recently with adding this functionality to my bitcoind front-end.

Those of you who aren't running full nodes will not find this useful. However, if you do have a full node, you can do all of this locally without external API requests. It may even be faster (I have not tested chainsnort, so I do not know.)

https://github.com/esotericnonsense/bitcoind-ticker

At the moment this is just a proof of concept and does the absolute bare minimum required. No highlighting or anything fancy, just spews out transactions to the terminal. The idea is there though.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

I, Atelopus Zeteki <azeteki@safe-mail.net> am the owner of the public key:

0x47DA40099E00994C Daniel Edgecumbe <daniele@esotericnonsense.com>

and the GitHub username:

https://github.com/esotericnonsense

and the domain name:

https://esotericnonsense.com

The Daily Telegraph 2016-05-02 'Families in dark as doctors let patients die'
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCgAGBQJXJ1KsAAoJEOK9FOwsfUWP+CoQAKKyh/8MUlZxsahH+KjJG5Kf
3z/jcMZVJyyOD/Pc7Gd6Iy+LBWVenmUdwtRKeuzIENX/aZ0vU7bnXqNBPG3NgcQE
tMbewxEyMbdGUi7LqKdTB1yFiX+BODXL3v5WpUAGfhjbxNe8kXCErPp1iUQhErlc
Z5zK9+cmiCTqPQScaUsu4kQu0QsFILWa7wjs23OTMJRjxSYF1aXiqircwgsMg7Q9
9f6Xd9zGLL2mCFVB2rFvi+ROlOcMVZ2/DfVwydW0mbBe9u67AEbZjQ65Pjy5tkbJ
hByn10B55+sYYdbypFUTzVgmyhW8N3Lzz5yEkKYZHhCiNdP+uJaqDW7QV8f0M6uB
MMCpz+hppFQWdmq63VCbTE/GqbhDczeRr8h4UpJ5LNw9cWbPqrBIlbFLdzY7RWI6
nOP4fjce4YJHxCympo6ZP1PgZHLj6Yhdc5TcaSM+/cSQXF8BtmkdYFJHcpeZCch9
YWrGjpyxtr7X8+QCkXGGxlYN4AOoDk05CnT9ZS+tIonPLvTzF2RpOXJRH3wxNR74
Pr08SVDK95r7S9JJEknVGflKV++CeE0dtWBqseUBryKIqBNQKqSQmEjIuigWaCL7
nrOKngfI/m3GEu6nVtuUNnQZ22BDYgCyUV6b7qXCGAUhhuGzojnizNgls/UJpDzw
atwbohdtFZc5/BonemHP
=sBpY
-----END PGP SIGNATURE-----

Well, it's been a while!

A few small tweaks have been made to the repo.
Andrew Chen helpfully posted a fix for Bitcoin Core v0.12 in my absence, which has now been merged.

I have changed my GitHub username - a PGP signed confirmation of this with the same key as my original post here will be forthcoming (currently battling with importing the key).

- esotericnonsense/azeteki

Hello e1ghtSpace;

Consider the very small and simple case of a 1-bit key.

Combinations are as follows:
0
1

Now go up to a 2-bit key. You have 2 possible combinations of a 1 bit key, and 2 possible combinations of the new bit.
00
01

10
11

3-bit: 4 possible combinations of the lower 2 bits, and 2 possible combinations of the new bit. 8 in total.
000
001
010
011

100
101
110
111

Every time you add an additional bit, you have all previous possibilities, multiplied by 2 (as the first, new, bit can be one of two possible values).

Therefore a 1 bit key has 2 possible values; a 2 bit key 2*2 = 4 possible values; a 3 bit key 2*2*2 = 8 possible values; and so on and so forth, until an N-bit key has 2^N possible values.

You can test this if you wish. Try counting from 0 to 31 in binary. See what you find.

It seems you've mostly figured this out now, but let's deconstruct a bit.


I think a better way to understand this would be to consider an orphaned block to have previously been a valid block, but no longer.

If the longest chain is at 300K, and there's e.g. an orphaned block 200,001, at this point it's almost totally irrelevant.

Any transactions included within it have almost certainly either been relocated into another block within the main chain, or simply replaced (inputs spent out from under them).

Consider that a transaction that exists only in orphaned blocks, but not in the main chain, is an unconfirmed transaction. If it never makes its' way into the main chain, it 'didn't happen', as far as the canonical state is concerned. The recipient, if they're aware of such, has good incentive to rebroadcast until the tx is confirmed again.

Aside from curiosity purposes, the only reason I can think of to worry about non-main-chain blocks is very close to the tip of the main chain - it's possible that they have not made it in to the main chain yet.

Updated for 340709 (2015-01-27 15:54:27).

I have not yet bothered to create a 2015 page as there wouldn't really be much data yet. For now 2014 follows through to January.

As a side note, if anyone happens to have a host I could use that would be very much appreciated. Bandwidth usage should be utterly minimal.
The entire site including all graphs is about 500KB at present.

Hi JackH. Sorry for the delay in responding.

I have not added account tracking because my impression based on hanging around IRC is that accounts are a deprecated feature and may well end up being removed completely at some point.

Feel free to hack away and submit pull reqs if you are that way inclined though.

(The transaction/wallet side is really more of a curiosity than anything else to me personally at present.)

Thank you vertoe.

If anyone has used this over SSH or similar some feedback would be useful. I have only tested locally. Others have suggested ideas on how to emulate latency that I'll be looking into later.

I plan to introduce more configuration options at some point to avoid the need for people to go in and change hardcoded stuff.

For example, at present the RPC backend updates every 2 seconds and the monitor view every second. Over a high latency link this might be too often. You can change this if you know where to look but that's hardly ideal.

I would also appreciate feedback from anyone who has tried this on an RPi, Cubox or similar machine (e.g. top/htop cpu usage stats, if it's usable, etc.) That would help me to get an idea of where to focus development.

v0.0.22.

Added some basic logging functionality which should help for debugging purposes and identify areas where speedups can be made. (getrawmempool vs. getnetworkinfo is a big one at present).
For now the loglevel is hardcoded in rpc.py. Right at the top is a function with 'if loglevel > x'.

loglevel 0: (default) Nothing.
loglevel 1: Regular updates, new blocks, halt request.
loglevel 2: All RPC requests.
loglevel 3: Time taken for bitcoind to respond to each RPC request.

Example at log level 2:


Screenshots have been updated with new functionality, see the second post in the thread or github.

You may find the tool in my signature (bitcoind-ncurses) useful.

I have not performed any testing with btcd however other users have reported success with some tweaking (I believe all that is needed is to produce a config file that is formatted in the same way as Bitcoin Core).

It refreshes the number of peers at regular intervals. There is a peer display screen which shows all peers individually however this does not auto-refresh - may do so in the future.

Screenshot:

Yes. Most (all?) of these huge fees are caused by user error. People mucking around creating raw transactions, or coding bad wallets.
When creating a transaction by hand, the difference between inputs and outputs is given to miners as a fee.
A naive hacker can easily use a 50BTC input and assume that paying 3BTC to an address results in 47 remaining at the old address. Not so - the remainder of 47 goes off to mining land.

Updated charts for block 334725.

The most interesting changes recently to me are the leveling off of difficulty, and the continued increase in average block size. Blocks are roughly 375KB on average now, up from 225KB six months ago.

v0.0.21 - basic addition of 'getchaintips' for users running master

Very small update that hopefully explains itself for users on git master. I imagine the feature will make it in to v0.10. Should fail safe on non-git though I've not tested.

I forgot to mention the reason why it seems these blocks must constitute part of the main chain and cannot form some sort of asynchronous/synchronous 'chain on the side'.

It seems to me that it cannot be the case that 'chain #2' falls out of sync with chain #1, otherwise one or both of the chains become useless as a temporal ordering.

So the miners on each chain seemingly must be aware of the transactions on the other chain, which effectively means that you do have a single chain in which the 'canonical status' flips from #1, to #2, and back to #1 - but in order to trustlessly know which chain is newer, the prev block hash must be included, which means that you effectively have one chain.

If there's something I am missing please feel free to trash me.

Your proposal seems to lack a number of details necessary to really evaluate it.

I am assuming for the sake of discussion that these 'mini blocks' are considered equivalent to a 'big block' as regards their canonical status within the chain. That is, they would build upon previous blocks and be built upon, they would be considered as 'one confirmation', and so on.

Block 100 may reward 25BTC, block 101 may reward tx fees of 0.25BTC, block 102 tx fees of 0.3BTC, block 103 25BTC, and so on.

First of all, they would necessarily have to use a different algorithm than the big blocks. Otherwise the big miners would simply mine them as well. The competitive process of mining results in those who have the most efficient (GH/J or ultimately GH/$ i.e. low cost of energy region) hardware outcompeting others.

To illustrate this clearly with arbitrary numbers. Assume 1BTC is worth a bit less than $1000 for the sake of simplicity.
I am a miner - I would, disregarding risk, be willing to spend around $25000 to mine a 25BTC block.
For a 'tx fee' block, let's say it contains 0.25BTC - I would therefore be willing to spend around $250. I think this should be apparent.

A proper mining operation (latest hardware, or located in a cheap energy region, etc) will be able to produce more hashpower for that $250 and hence outcompete all others. edit: Assumption being that there is no reason other than goodwill or anonymity for someone to spend >$250 incl all overheads to mine $250 worth of BTC)

So it must be based on a different algorithm for that reason.

It must also be based on a different algorithm for the following reason - it is hugely problematic for the network to have 'easy/low reward blocks' that could be misconstrued by users as representing a full confirmation. It would be utterly trivial for someone to 51% attack the 'mini chain' if it shared an algorithm with the main chain.

There are other issues floating around in my head right now (if the blocks are not more difficult, but still have canonical status within the chain, they weaken it rather than strengthening it due to the ease of attack) but I don't really want to rip this apart completely - can you think of solutions to these issues?

This is rather interesting but the RPC server should not ordinarily be exposed outside of a trusted network. Certainly not with an unencrypted wallet.
This was one of the main reasons behind me creating my terminal based frontend.
The approach should be to connect using a secure tunnel like SSH and interface with the Bitcoin Core daemon from there.

Ignoring that, I would take issue with the claim that it would take around an hour to brute force the password if on same network.
I can't say I've tried but you are claiming that you can get off over a million authentication attempts per second over a network. (4294967296/3600).
Just sending a ten byte auth request would make that 10MB/s sustained.

The RPC server is not especially fast. I have not tested but it would not surprise me if you struggled to get a few hundred auth attempts per second on a local machine. That would put you at over a month. If anyone has the time it would be interesting to see how quickly you can fail auth and try again.

I don't wish to speak for the core developers here but I would not be surprised if there are numerous vulnerabilities in the RPC server - it is likely not intended to be used with unsanitised input.

As shorena says.

Strictly to my understanding the coinbase can (presently) be anything _up to_ 25 + sum of all transaction fees.

A miner can produce a block with a coinbase of 5BTC and it will be valid. It is just not really an economically sensible thing for them to do.

Updated for block 332749 2014-12-03 21:29:57.
Apologies for the delay. I have been away attending to various worldly matters.
Some of the charts may look a little odd as I have not had time to adjust the axes.

I have nothing to add here as someone who has (at best) a novice level of C/C++ knowledge.
But I want to thank you both for adding to that.
These are the issues that everyone should have on their minds when writing sensitive crypto code.

As Gavin says in a round-about way; we can only do our best with the time we are given, but we do have a responsibility here.

I'm sure I'm not the only one that sometimes wonders how software manages to function at all. It often feels as if there are thousands of potential entry vectors on my machine and just one is enough. But that doesn't mean it's not worth trying our best.

edit: Just noticed this gem from gmaxwell's post:
.

Emphasis on the latter point there, even if none of the technical points sink in.
You're engineering a car. Alice does not necessarily know and cannot be expected to know the potential failure points.

gmaxwell,

I've missed being able to stay in touch with the field. Glad to see that bitcoin has not yet died.

getchaintips looks quite interesting. If I'm interpreting the source comments correctly (I can't compile and test right now) then it matches a feature I was intending to add to bitcoind-ncurses - monitoring forks and their eventual resolve.


Indeed.
There are a few different approaches to this whole business that I can think of.

The first is to rely on bitcoind to perform the wallet functionality. Tell bitcoind to send some coins, and bitcoind consults its' wallet.dat and so forth.
Least likely to go wrong, but limited in scope.

The second and more ambitious is to effectively produce a standalone wallet that uses bitcoind to connect to the p2p network only. In this case bitcoind-ncurses would have its' own 'wallet.dat' format, produce its own keys, do its' own coin control, and so on and so forth. Much more 'fun' for me, and also achieving the secondary aim of producing a viable split between the 'node' that is bitcoind and the 'wallet' that is bitcoind, but requiring that I really know what I'm doing.

What I would need to do to feel comfortable with either solution is to spend a lot of time studying how bitcoind achieves its' wallet functionality and also ensure that Python is even capable of producing a reasonably secure wallet (which would entail a fair amount of crypto-study).

To elaborate on the latter point - take a basic function like 'walletpassphrase'. This needs to be handled very carefully. You can end up in a situation where the passphrase sits around in RAM for the rest of all eternity after being used. Or perhaps the size of the program's RAM usage increases by some predefined way based on passphrase content. Maybe the query takes a different amount of time depending on the length of the password, etcetera. Some of these probably matter as side channel attacks and others may not. If I am to produce a wallet tool for general consumption then I have a moral obligation to do everything within my power to ensure that it is safe because people can and will use it to store lots of money.

It makes me very happy that people have found my work useful so far. Finding some solution to fund the roof over my head whilst also pursuing this sort of project remains the challenge we all struggle with.

I am glad to hear that you found it useful.

As of late a number of life issues has meant that I've had to take a step back from Bitcoin for the time being.
With a little luck I may be able to continue development (in addition, I would like to branch outside of bitcoind-ncurses and pursue a number of other ideas).
The world will decide this in time.