Quote from: Dogecoin on August 09, 2015, 08:28:26 PM

As we're closing on a beta release for Dogecoin Core 1.10, I wanted to talk about where 1.9 went to, why we haven't had a major release in 11 months, and what we're doing differently in future. This is a long post, but I swear it's worth reading in full.

First of all, important security announcement: If you're using brain wallets (this won't be many of you, but want to ensure we catch anyone who is), stop, and move your funds right now. There was a security talk at DEF CON which basically explained how much their security is broken, more detail at https://rya.nc/cracking_cryptocurrency_brainwallets.pdf . For anyone who's unsure, brain wallets are where you pick a set of words and use them to generate a wallet, such as bip32.org (I'm not linking that) lets you do. If you have been given words by a random process (i.e. Multibit HD, Electrum, Trezor, Ledger), these are AFAIK fine, it's just manually chosen words that are a disaster waiting to happen.

Next, there's a Bitcoin village, at Chaos Communication Camp next weekend, and while the core developers can't attend (we're doing dull day job things instead), Dogerain's developer will be there, and they're organising a video hangout with the Dogecoin core devs. Not sure if others can attend remotely, but if you're at the camp we'd love to get to talk to you!

Right, back to 1.9; Dogecoin Core 1.9 was going to be 1.8 with the Bitcoin Core 0.10 changes merged in. The same process was used to make Dogecoin Core 1.8 from 1.7 with Bitcoin Core 0.9, so we knew what we were doing. With almost 1,300 commits to review and apply it would take a while, but in theory was straight forward enough. A spreadsheet was created to track progress amongst the developers, and in January we set out to start merging.
At this point we discovered several things:

* Some patches from 0.10 had been merged in early and out of sequence, so we had to avoid merging them twice.
* A lot of the changes were less readily compatible with Dogecoin Core than we expected.
* 1,300 is really a lot of changes

As time dragged on, we gained further assistance (Sporklin, this means you) in preparing merged commits, and I made several attempts at automating much of the process. Around March we started struggling with keeping development motivation up, and pace faltered, with Sporklin taking on much of the charge to keep work continuing. In June, we were about half way, and Bitcoin Core 0.11 hit release candidate, and at that point we realised this wasn't going to work.

So, Dogecoin Core 1.10 is a rebuild. We've started with Bitcoin Core 0.11 as a base and then manually re-applied the Dogecoin changes. This makes a lot of sense, in as much as they're a smaller set of changes (and less invasive by design), but does mean that we risk losing subtle tweaks to the code (which is what the beta period is intended to help catch). Most of the changes have been totally rewritten to make them simpler to apply, and better fit in with the hugely revised code base. We also see a significant number of changes in the strings with Dogecoin, so previous improvements to translations cannot necessarily be used as-is, and when we hit beta we'll be looking for help with updating translations.

The loss of motivation is something we need to be more aware of as a risk; while the Dogecoin developers are not doing this to try getting rich, that doesn't mean that there's no motivation required. We enjoy the challenge and opportunity to work with interesting technology, and based on that it's important that we ensure the work does have its interesting parts in amongst just getting stuff shipped.

Looking ahead to future work:
* We'll do a full rebuild once per year, potentially twice, to keep us close to the Bitcoin Core code and ensure compatibility.
* In between these rebuilds, we'll merge in changes where feasible.
* To avoid Dogecoin and Bitcoin diverging, we'll push new features and fixes into the relevant upstream project where practical. We avoid divergence because it makes it harder to update, and requires custom code to adopt Dogecoin compared to Bitcoin.
* Dogecoin Core will be promoted as the reference base for other Scrypt-based altcoins. This happens already, and we get fixes from downstream (i.e. Fractalcoin caught that the fork detection code is too sensitive) as a result.

I know there are those who wish to see Dogecoin split further from Bitcoin, but there's just far too much effort being poured into Bitcoin, and too much available expertise from working with them, to ignore.

On a related note, bitcoinj 0.14 now has all of the changes to make it work with the libdohj wrapper library. Patrick's been testing libdohj, and so far mostly it seems to work well (there's an issue with the advertised network protocol version that I need to fix, but apart from that so far so good). There's a similar model for python-bitcoinlib and python-altcoinlib, although I need to dust off python-altcoinlib somewhat.

There's tons more I could write about HD wallets, user defined consensus or Ledger wallet support, but I think that's quite enough for today. There will be an interim update for Dogecoin Core 1.10 work around next weekend, hopefully a beta around the same sort of time, and the next full update post should be on the 23rd or thereabouts.

How about building a separate branch for doge specific codes?

Any new changes specific to doge should be merged to this separate branch first. You will only merge these changes to master when you want new client version release, while you may add a tag to make it clear on master these changes are doge-specific and the rest are not.

So even if the next Bitcoin v0.12 may contain 2000+ new commits, you won't easily lose moviation to rebuild doge client again.