Quote from: marilyn4325 on December 23, 2017, 07:28:23 PM

Two years ago I formatted my harddisk and installed Windows 10 on it. Before this I did a backup, but unfortunately the backup was broken. So I lost my wallet.dat, with a few Bitcoins in it. I could restore some files with RStudio, and I had older backups for the rest, but seemed to be that the latest wallet.dat was already overwritten, and I frequently add new addresses. So I gave up, not a big deal, maybe $200 lost. But I didn't use the harddisk and bought a new one.

Fast forward to December 2017: Now a few Bitcoins is some serious money, so I decided to give it another try. I tried any option I could find in RStudio, checking the dozens of filesystems it reported after scanning it for hours (only a few where valid from previous installations), but I couldn't restore it. Ok, this needed some more work.

My assumption was, that the file headers were broken, so I wrote a small C program myself, which scanned the whole harddisk for the wallet.dat signature (testing for the first 16 bytes). The filesystem was NTFS, which has 4k sector sizes and a file starts always at sector start, if I understand it correctly, which makes things easier. Also usually if there is enough space, contiguous sectors are used to save a file. My hope was that somewhere I could find old version of the wallet.dat, but not too old that the new keys were missing.

This is the very simple and straightforward scan program I hacked together:

Code:

#include
#include
#include
#include

uint8_t buf[4096];
char filename[1000];

uint8_t search[] = {
0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x62, 0x31, 0x05, 0x00
};

int main(int argc, char** argv)
{
uint64_t pos = 0;
FILE* f = fopen("/dev/sdd", "rb");
FILE* w = NULL;
int walletNumber = 0;
int walletIndex = 0;
uint64_t max = 1000000000000ULL;
while (1) {
int c = fread(buf, 1, 4096, f);
if (c != 4096) break;
if (!w) {
if (memcmp(search, buf, 16) == 0) {
sprintf(filename, "wallet%i.dat", walletNumber++);
walletIndex = 0;
w = fopen(filename, "wb");
printf("found: %" PRIu64 "\n", pos);
}
}
if (w) {
fwrite(buf, 1, 4096, w);
walletIndex++;
if (walletIndex == 256) {
fclose(w);
w = NULL;
}
}
pos += 4096;
}
fclose(f);
return 0;
}

I used it on Linux as my host system and the old harddisk was visible as /dev/sdd (you can see this with dmesg). I compiled it with "gcc -O3 scan.cpp -o scan" and started it with "sudo ./scan", and a few hours later (it was a 1 TB harddisk) I got a wallet0.dat to wallet9.dat, each 1 MB in size (it doesn't matter if there is crap after the wallet data). This was a nice start Grin

Then I tried to copy it to a wallet.dat of a current Bitcoin installation, but most of the time it said the wallet was corrupt, once it even crashed at start and when it said it could salvage some information, no keys were in it.

My rescue was https://github.com/joric/pywallet This program could decode all files and output it in JSON format. It needs the wallet.dat in a bitcoin-qt installation in the .bitcoin directory. I knew one of my old addresses, so I wrote a script which did test all files (actual key changed) :

Code:

for i in $( ls wallet*.dat ); do
echo item: $i
cp $i .bitcoin/wallet.dat
./pywallet.py --dumpwallet --datadir=.bitcoin | grep -i 12QDRXssT63Pv5KTGBN2kyAvfLW3s7jxBs
done

The output looked like this:

Code:

item: wallet0.dat
item: wallet10.dat
ERROR:root:Couldn't open wallet.dat/main. Try quitting Bitcoin and running this again.
item: wallet11.dat
ERROR:root:Couldn't open wallet.dat/main. Try quitting Bitcoin and running this again.
item: wallet1.dat
WARNING:root:encrypted wallet, specify password to decrypt
item: wallet2.dat
item: wallet3.dat
ERROR:root:Couldn't open wallet.dat/main. Try quitting Bitcoin and running this again.
item: wallet4.dat
Traceback (most recent call last):
File "./pywallet.py", line 1706, in
main()
File "./pywallet.py", line 1683, in main
read_wallet(json_db, db_env, True, True, "")
File "./pywallet.py", line 1556, in read_wallet
parse_wallet(db, item_callback)
File "./pywallet.py", line 1287, in parse_wallet
for (key, value) in db.items():
bsddb.db.DBPageNotFoundError: (-30986, 'BDB0075 DB_PAGE_NOTFOUND: Requested page not found')
item: wallet5.dat
ERROR:root:Couldn't open wallet.dat/main. Try quitting Bitcoin and running this again.
item: wallet6.dat
WARNING:root:encrypted wallet, specify password to decrypt
"addr": "12QDRXssT63Pv5KTGBN2kyAvfLW3s7jxBs",
item: wallet7.dat
WARNING:root:encrypted wallet, specify password to decrypt
item: wallet8.dat
ERROR:root:Couldn't open wallet.dat/main. Try quitting Bitcoin and running this again.
item: wallet9.dat
ERROR:root:Couldn't open wallet.dat/main. Try quitting Bitcoin and running this again.

So the address was in wallet6.dat, success! I then used "/pywallet.py --dumpwallet --datadir=.bitcoin --password=mysecrectpassword > keys.txt" and I got all my keys back. In the bitcoin client I could import it with importprivkey (don't forget the "false" parameter as the last parameter, to avoid rescanning after each import, if you import multiple keys) and after the final rescan, I got my Bitcoins back. One day work for like 2 Bitcoins, which I already sold, that's a nice hourly rate Cool

Maybe this will help some other people as well. In case you rescue a lot of Bitcoins, I would really love it if you would send me some to 1HagWYdLaFRbXwUfzbuWzcDc4WzmsRGqg7.

Marilyn wishes you a merry Christmas, a merry Christmas, And a happy New Year!

That was a roller coaster of a ride Shocked

, you were very lucky and that was a great christmas present for you. If only I could have found old wallet dats I mined, but of course I am a late adopter in terms of you all. I would love to one day say that I made a great decision to invest in bitcoin, again great job to you and I am glad you found the very thing that has made you happy this year! Merry Christmas!

arjen20

Posts

Merits