Quote from: Trent Russell on January 09, 2016, 06:03:33 PM

I was working with the code for Clams and thought of how I might be able to modify it to provide the network infrastructure you need for Qeditas. Clam transactions have a "clamspeech" part which can hold arbitrary data. This could be used (or abused?) to have dummy Clam transactions whose main content is a clamspeech part carrying Qeditas data (txs, blocks, ctrees, etc). In addition to giving a way of sharing the data between peers, the dummy txs with the Qeditas data can be stored in the local leveldb tx database.

I forked Clams, made some modifications and renamed it "qednet". It's on github with a "trdev" branch that has the actual modifications:

https://github.com/trentrussell/qednet/tree/trdev

I also put the Qeditas git repo back on github. There's also a "trdev" branch there, but I haven't started modifying anything yet:

https://github.com/trentrussell/qeditas

I'm thinking Qeditas (ocaml) can start qednet (C++) in a process. Qednet will connect up with peers and share inventory/data (this works already). Qeditas will be notified when there are new Qeditas transactions or Qeditas blocks. Qeditas can then ask qednet (via rpc commands) for the actual data which Qeditas can verify and organize (e.g., finding the "best" current block/ledger).

I should probably say that I'm not really a C++ programmer. The parts of the qednet code that didn't directly come from Clams was mostly copy-paste-modify from code in Clams. There is also some strange bug the first time I try to run qednetd. I'd expect it to make the .qednet directory and then advise the user on making a qednet.conf file. Instead the first time I run it it makes the .qednet directory and then dies with a seg fault. The second time I run it it warns that I need to -reindex. I don't actually need to -reindex (there are no blocks). Instead I run it a third time and it prompts me to make a qednet.conf file. After making the qednet.conf, I can run it a fourth time and it starts working. If this bug sounds familiar to anyone let me know.

I'm not sure any of this will work well enough to really start Qeditas, but at least it might keep the Qeditas project alive with a test network.

PS: To check if the first block is valid, I need the ctree corresponding to the initial ledger. I guess one node needs this and then it can get shared. @Bill: You posted a link to the initial distribution, but it's text files with address hashes and amounts. Do you have the ctree itself available somewhere?

This sounds like an interesting approach. I will need to look more closely before I can say more.

Regarding the initial ctree, I do have a binary version. It consists of approximately 100,000 files each of which contains part of the ctree. The files range in size from 74 bytes to 769042 bytes. However, the large one is an outlier. The next largest files are 146365 bytes and then 57852 bytes.

A compressed directory with all these files is available from mega here:

https://mega.nz/#!tgghSCKZ!aXnBE1d7nI65quPpiGvgLOOrTg3ODeuSud1BqAxvLkQ

It is roughly 250MB total.

There is Qeditas code for loading these files and obtaining the part of the ctree they represent. In particular, see the function get_ctree_abbrev. You will first need to set ctreedatadir:

Code:

Config.ctreedatadir := "/fullpathto/initctrees"

The initial ledger root is af8de840af01805f1dbe9f1312c388efeea1e619. An abbreviation for a ctree with this ledger root is named 0b7f6f55f3b4b05a9a7bd99d01a087d799e01b9c. This can be loaded and viewed as follows:

Code:

let cinittop = Ctre.get_ctree_abbrev (Hash.hexstring_hashval "0b7f6f55f3b4b05a9a7bd99d01a087d799e01b9c");;
val cinittop : Ctre.ctree =
Ctre.CLeft
(Ctre.CBin
(Ctre.CAbbrev
((1442101457l, 180266486l, 913955555l, 2050428135l, -929962841l),
(2055486346l, 214952729l, -1159736634l, -1343903178l, 337615324l)),
Ctre.CAbbrev
((-1653840840l, 739647671l, -1962027568l, 372448381l, -1899755430l),
(1623437078l, 1755382547l, -619214889l, -762676765l, 1420032299l))))

This represents the top two levels. The first two bits indicate the type of address: p2pkh, p2sh, term, publication. Since there are initially no terms or publications, the right child of the root is empty. This is why cinittop starts with "CLeft". The next "CBin" splits into the left (p2pkh) and right (p2sh).

If we simply look at the left child, it has root "(1442101457l, 180266486l, 913955555l, 2050428135l, -929962841l)" and abbrev hash "(2055486346l, 214952729l, -1159736634l, -1343903178l, 337615324l)". Note that hash values are represented by 5 32-bit integers. You can also see it in hex as follows:

Code:

Hash.hashval_hexstring (2055486346l, 214952729l, -1159736634l, -1343903178l, 337615324l);;
- : string = "7a843b8a0ccfeb19badfd2c6afe5aa36141f99dc"

There is a file named 7a843b8a0ccfeb19badfd2c6afe5aa36141f99dc holding the ctree which hashes to this value. It can be loaded as follows:

Code:

let cp2pkhinittop = Ctre.get_ctree_abbrev (2055486346l, 214952729l, -1159736634l, -1343903178l, 337615324l);;
val cp2pkhinittop : Ctre.ctree =
Ctre.CBin
(Ctre.CBin
(Ctre.CBin
(Ctre.CBin
(Ctre.CBin
(Ctre.CBin
(Ctre.CBin
(Ctre.CBin
(Ctre.CAbbrev
((1966404489l, 484598006l, 138518841l, 1976927092l,
-869398977l),
(1262074113l, 53533383l, 1018187993l, 684848879l,
1620505634l)),
...

This corresponds to the p2pkh contents split into 256 "abbrevs" based on the first byte of the p2pkh address. I'm only showing the first abbrev, corresponding to p2pkh addresses with an initial 0 byte. We can load this subtree as well:

Code:

let cp2pkhinit00 = Ctre.get_ctree_abbrev (1262074113l, 53533383l, 1018187993l, 684848879l, 1620505634l);;

This will give a tree with 8 levels again, splitting based on the second byte of the address. The abbrev hash for the left-most subtree is (1602475596l, -1426117715l, -1600090211l, -2046806429l, 138862169l)
(i.e., 5f83d64caaff2bada0a08f9d860036630846de59). This subtree contains all the contents of address for which the first two bytes are 0.

Code:

let cp2khinit0000 = Ctre.get_ctree_abbrev (1602475596l, -1426117715l, -1600090211l, -2046806429l, 138862169l);;

I hope this helps.