Quote from: goatpig on November 20, 2018, 06:30:42 PM

Quote

Once it writes to a blk file, it never checks files lower than that anymore. That is a RAM variable though, so it gets set back to zero each time a node is restarted.

Why would it need to check for previous files appends on new blocks? If the issue only happens on fresh node start (when free space in older block files is eligible), as long as Armory sees that one new block, it will work.

I am not sure if they fully thought out the logic. I may have missed something when looking at the code on core that protects against the issue.

I think it is a pretty bad policy no matter what. It's not like keeping all the blk files at the same size actually matters, and keeping nearby blocks in the same file is likely a help.

They want a way to fill in spaces in the old block files but once a file overflows since last restart, they just don't bother with that one anymore.

The sequence is

- on startup set nLastBlockFile to 0
- when a new block arrives scan for a blk file with space starting at index = nLastBlockFile
- Once it jumps to a new file, it sets nLastBlockFile to that value

This means that nLastBlockFile never decreases, so once the node is running for a while it only writes to the latest blk file.

On the next reboot, different spaces could end up being filled.

Quote

Armory does not expect blocks to appear in order in files. It will check block data on disk from the last known block to have extended the chain. Therefor if you have up to blk01000.dat and the last top is in blk00997.dat, it will check #977 to #1000 on every new block notifications until it finds a block that extends the chain again.

Cool, so re-scanning problematic blk files wouldn't be an issue. The only question is detecting them.

It looks like c++ doesn't have a portable way to get the last modified time.

The assumption that later numbered blk files contain later data is violated by the code. In that situation, blk #800 could be updated and no amount of scanning after 977 will fix things.

Quote

The ultimate solution would be blocks over P2P, but idk what kind of mess BCH will be by the time I'm done with that. Chances are Armory won't be compatible anymore at that point.

I don't think it is only restricted to Bitcoin Cash. The problem is much more likely to be exposed by the variable block sizes, but I think it is possible even if they are mostly stable blocks. You could get a 950kB space that doesn't get filled for a while and then someone is unlucky that they start their node just before a block that can fill that space arrives.

It might actually "settle down" once it has synced and then been restarted a few times. This would "fill in" many of the spaces. As long as the first block found after a restart is bigger than any of the spaces, then it will write to the file with the highest number and that jumps nLastBlockFile to the max value.

I think the workaround of wiping the armory database is probably acceptable, since the problem is rare and doesn't seem to affect the main chain.