Search content
Sort by
Showing 20 of 261 results by belcher
If you use Electrum Personal Server and send transactions, please help test the new Tor broadcasting feature which improves privacy
https://github.com/chris-belcher/electrum-personal-server/issues/130
https://github.com/chris-belcher/electrum-personal-server/issues/130
Quote
Another demonstration of the fragility of blockchain analysis.
While I realize you just meant it as a light "fun fact", I think it's worth pointing out that walletexplorer is very primitive and semi? unmaintained -- and you won't be able to trick any serious analysis tool with a coinjoin like that. [Although coinjoins can do an amazing job at tricking them! But you really need the coinjoin to look like a normal transaction for that]
You're right that is fairly primitive but many people still use it and it has some influence. During the QuadrigaCX exchange hack affair in early-2019 some people used walletexplorer to find that exchange's hot wallet, some of the transactions go to and from the CoinJoinMess cluster (which then was called MtGoxAndOthers). When this was found a bunch of people were posting that QuadrigaCX was receiving money from MtGox(!) They carried on until they were informed that it's only the coinjoin cluster.
I wouldn't say its completely trivial to detect that something is odd with the coinjoin bounty payout. The inputs use multiple address types, but Samourai wallet and Bitcoin Core also sometimes do this so it's not evidence of non-coinjoin behaviour. Also there are many equal-valued outputs, but the transaction doesn't match the style of JoinMarket or Wasabi transactions (there are far more equal-valued outputs than inputs for example). It would definitely be interesting to see what the more developed tools say about it.
Fun fact: because the CoinJoin bounty payout transaction to JoinMarket and Wasabi wallet was itself a coinjoin transaction with specially chosen inputs, the wallet clustering site walletexplorer.com now thinks that the coinjoin bounty address belongs to the largest wallet cluster (which used to be called MtGoxAndOthers and is now called CoinJoinMess)
https://www.walletexplorer.com/wallet/CoinJoinMess?from_address=3M8XGFBKwkf7miBzpkU3x2DoWwAVrD1mhk
The cluster contains nearly 9 million transactions and over 3.5 million addresses, including of course the CoinJoin bounty multisig address itself. Another demonstration of the fragility of blockchain analysis.
https://www.walletexplorer.com/wallet/CoinJoinMess?from_address=3M8XGFBKwkf7miBzpkU3x2DoWwAVrD1mhk
The cluster contains nearly 9 million transactions and over 3.5 million addresses, including of course the CoinJoin bounty multisig address itself. Another demonstration of the fragility of blockchain analysis.
Re: Payment Channel Payouts: An Idea for Improving P2Pool Scalability
⭐ Merited by ThatRandomDude (1)
A way to reduce the liquidity requirement of hubs in lightning p2pool, potentially by three orders of magnitude. The first part of this post briefly summarizes the original lightning p2pool proposal in a slightly new way, the second part introduces this new idea.
The original proposal had this as the script of the coinbase output.
Control of the coinbase is divided, between the hub and the successful hasher, in a complicated way.
In the case where everyone cooperates, branch (1) is used to send the coinbase to the hub. (But only after the hub has sent all the hashers the correct amount of money according to the state of the sharechain)
The successful hasher could defect, either by becoming unresponsive or trying to hold the hub to ransom on a Mutually Assured Distruction (MAD) basis. To avoid that, branch (2) allows the hub to collect the money on its own. It must reveal the preimage X which allows every other hasher in p2pool to collect their rightfully-owed money.
The hub could also defect, by being unresponsive or holding the successful hasher to ransom via MAD. Branch (3) avoids this by providing a timeout (of 1 week in this example) after which the successful hasher can collect the entire coinbase. This punishes the hub, providing a disincentive to doing this.
Now, we remind ourselves of the Schnorr signature + taproot + MAST proposal, which will probably be added to bitcoin one day via a soft fork. Using Schnorr signatures it allows any N-of-N multisig to take up the same blockchain space as a single signature. MAST allows any syntax tree with N branches to be efficiently committed to, spending via one branch only requires log2(N) data to be added to the blockchain. Taproot is a way to combine Schnorr and MAST which avoids having to reveal any MAST merkle proof if it never gets used.
------
With all this mind, we propose another script of the coinbase output, this time with two hubs. The hubs have preimages X and Y. The "OR" statements are left out for readability, branches are numbered and successful hasher is abbreviated to SH. The hubs both periodically broadcast hash-locked channel updates, same as in the original proposal.
The proposed coinbase script for the situation with 2 hubs:
What we have is a branch covering every possible combination of cooperate and defect:
* Branch (1) is a 3-of-3 multisig for everyone cooperates, it is the most common situation.
* Branch (2) is when SH defects, it allows the two hubs to divide the money equally between them, but they must reveal the preimages so that every other hasher can get paid as well.
* Branches (3) and (4) are used when one hub defects, it allows SH and the other hub to take their money and punish the defecting hub by stealing it's money too.
* Branches (5) and (6) cover the situation where SH and one hub both defects, it allows the remaining hub to punish them the other hub and get the money (but it must reveal its preimage).
* Branch (7) covers the situation where both hubs defect, the SH punishes them both.
The most important branches are (1) and (2). In the vast majority of cases all the payment channels will be honestly updated and branch (1) will be used. All the other branches are required to make it impossible to hold anyone else to ransom via MAD. The OP_CHECKSEQUENCEVERIFY timeouts are staggered so at each step there is plenty of time to respond.
Branch (1) can be efficiently executed in a single Schnorr signature. All the other branches can be hidden in a MAST merkle tree, hidden in a taproot scheme.
This reduces the liquidity requirement because each hub only needs to own half a much bitcoins as before. Today's block reward is around 12.5 btc, under this idea each hub only needs to provide 6.125 btc. All the other incentives and procedures as the same as before.
--------
Next, let's extend the scheme to using 3 hubs. Each hub has preimages X, Y and Z. The coinbase script for that situation would be:
This is very similar to before. There are 15 branches now, most of which would be never executed and are hidden in MAST. Branches (1) and (2) are again the most important. As before, it's not possible for any entity or group of entities to hold anyone else to ransom via MAD. All the hubs can be punished if they misbehave. Each hub only needs to provide 3 times less liquidity as before.
If any branch but (1) is executed then the MAST proof size is only log2(15) = 4 hashes.
Left as an exercise to the reader, you can work out that with N hubs the number of branches for a similar scheme is 2^(N+1) - 1, which means the merkle proof for any branch except for (1) is of size N.
If we choose N = 10 to have 10 hubs, the most common branch (1) of the 10-of-10 multisig is still just the same size as a Schnorr single sig. For any other branch the merkle proof size is only 10 hashes. But each hub requires 10 times less liquidity.
If we choose N = 100 we have 100 hubs. As before, the branch (1) is just a single sig and each hub requires 100 times less liquidity.
We can also use the previous idea of using multiple coinbase outputs. We can have 10 coinbase outputs, each with N = 100 hubs, so that would be 1000 hubs altogether, each requiring 1000 times less liquidity than the original proposal.
1000 is more of a demonstration to show how well this scheme scales. In reality it would run up against other bottlenecks like the bandwidth requirement for all those channel updates.
The original proposal had this as the script of the coinbase output.
Code:
(1) 2of2 multisig of hub + successful hasher
OR
(2) hub pubkey + H(X)
OR
(3) successful hasher pubkey + OP_CSV 1 week
OR
(2) hub pubkey + H(X)
OR
(3) successful hasher pubkey + OP_CSV 1 week
Control of the coinbase is divided, between the hub and the successful hasher, in a complicated way.
In the case where everyone cooperates, branch (1) is used to send the coinbase to the hub. (But only after the hub has sent all the hashers the correct amount of money according to the state of the sharechain)
The successful hasher could defect, either by becoming unresponsive or trying to hold the hub to ransom on a Mutually Assured Distruction (MAD) basis. To avoid that, branch (2) allows the hub to collect the money on its own. It must reveal the preimage X which allows every other hasher in p2pool to collect their rightfully-owed money.
The hub could also defect, by being unresponsive or holding the successful hasher to ransom via MAD. Branch (3) avoids this by providing a timeout (of 1 week in this example) after which the successful hasher can collect the entire coinbase. This punishes the hub, providing a disincentive to doing this.
Now, we remind ourselves of the Schnorr signature + taproot + MAST proposal, which will probably be added to bitcoin one day via a soft fork. Using Schnorr signatures it allows any N-of-N multisig to take up the same blockchain space as a single signature. MAST allows any syntax tree with N branches to be efficiently committed to, spending via one branch only requires log2(N) data to be added to the blockchain. Taproot is a way to combine Schnorr and MAST which avoids having to reveal any MAST merkle proof if it never gets used.
------
With all this mind, we propose another script of the coinbase output, this time with two hubs. The hubs have preimages X and Y. The "OR" statements are left out for readability, branches are numbered and successful hasher is abbreviated to SH. The hubs both periodically broadcast hash-locked channel updates, same as in the original proposal.
The proposed coinbase script for the situation with 2 hubs:
Code:
(1) 3of3 multisig of hubX, hubY and SH
(2) hubX pubkey + H(X) + hubY pubkey + H(Y)
(3) SH pubkey + hubY pubkey + CSV 1 week
(4) SH pubkey + hubX pubkey + CSV 1 week
(5) hubX pubkey + H(X) + CSV 2 weeks
(6) hubY pubkey + H(Y) + CSV 2 weeks
(7) SH pubkey + CSV 3 weeks
(2) hubX pubkey + H(X) + hubY pubkey + H(Y)
(3) SH pubkey + hubY pubkey + CSV 1 week
(4) SH pubkey + hubX pubkey + CSV 1 week
(5) hubX pubkey + H(X) + CSV 2 weeks
(6) hubY pubkey + H(Y) + CSV 2 weeks
(7) SH pubkey + CSV 3 weeks
What we have is a branch covering every possible combination of cooperate and defect:
* Branch (1) is a 3-of-3 multisig for everyone cooperates, it is the most common situation.
* Branch (2) is when SH defects, it allows the two hubs to divide the money equally between them, but they must reveal the preimages so that every other hasher can get paid as well.
* Branches (3) and (4) are used when one hub defects, it allows SH and the other hub to take their money and punish the defecting hub by stealing it's money too.
* Branches (5) and (6) cover the situation where SH and one hub both defects, it allows the remaining hub to punish them the other hub and get the money (but it must reveal its preimage).
* Branch (7) covers the situation where both hubs defect, the SH punishes them both.
The most important branches are (1) and (2). In the vast majority of cases all the payment channels will be honestly updated and branch (1) will be used. All the other branches are required to make it impossible to hold anyone else to ransom via MAD. The OP_CHECKSEQUENCEVERIFY timeouts are staggered so at each step there is plenty of time to respond.
Branch (1) can be efficiently executed in a single Schnorr signature. All the other branches can be hidden in a MAST merkle tree, hidden in a taproot scheme.
This reduces the liquidity requirement because each hub only needs to own half a much bitcoins as before. Today's block reward is around 12.5 btc, under this idea each hub only needs to provide 6.125 btc. All the other incentives and procedures as the same as before.
--------
Next, let's extend the scheme to using 3 hubs. Each hub has preimages X, Y and Z. The coinbase script for that situation would be:
Code:
(1) 4of4 multisig of hubX, hubY, hubZ and SH
(2) hubX pubkey + hubY pubkey + hubZ pubkey + H(X) + H(Y) + H(Z)
(3) SH + hubY + hubZ + CSV 1 week
(4) SH + hubX + hubZ + CSV 1 week
(5) SH + hubX + hubY + CSV 1 week
(6) hubY + hubZ + H(Y) + H(Z) + CSV 2 weeks
(7) hubX + hubZ + H(X) + H(Z) + CSV 2 weeks
(8) hubX + hubY + H(X) + H(Y) + CSV 2 weeks
(9) SH + hubZ + CSV 3 weeks
(A) SH + hubY + CSV 3 weeks
(B) SH + hubX + CSV 3 weeks
(C) hubZ + H(Z) + CSV 4 weeks
(D) hubY + H(Y) + CSV 4 weeks
(E) hubX + H(X) + CSV 4 weeks
(F) SH + CSV 5 weeks
(2) hubX pubkey + hubY pubkey + hubZ pubkey + H(X) + H(Y) + H(Z)
(3) SH + hubY + hubZ + CSV 1 week
(4) SH + hubX + hubZ + CSV 1 week
(5) SH + hubX + hubY + CSV 1 week
(6) hubY + hubZ + H(Y) + H(Z) + CSV 2 weeks
(7) hubX + hubZ + H(X) + H(Z) + CSV 2 weeks
(8) hubX + hubY + H(X) + H(Y) + CSV 2 weeks
(9) SH + hubZ + CSV 3 weeks
(A) SH + hubY + CSV 3 weeks
(B) SH + hubX + CSV 3 weeks
(C) hubZ + H(Z) + CSV 4 weeks
(D) hubY + H(Y) + CSV 4 weeks
(E) hubX + H(X) + CSV 4 weeks
(F) SH + CSV 5 weeks
This is very similar to before. There are 15 branches now, most of which would be never executed and are hidden in MAST. Branches (1) and (2) are again the most important. As before, it's not possible for any entity or group of entities to hold anyone else to ransom via MAD. All the hubs can be punished if they misbehave. Each hub only needs to provide 3 times less liquidity as before.
If any branch but (1) is executed then the MAST proof size is only log2(15) = 4 hashes.
Left as an exercise to the reader, you can work out that with N hubs the number of branches for a similar scheme is 2^(N+1) - 1, which means the merkle proof for any branch except for (1) is of size N.
If we choose N = 10 to have 10 hubs, the most common branch (1) of the 10-of-10 multisig is still just the same size as a Schnorr single sig. For any other branch the merkle proof size is only 10 hashes. But each hub requires 10 times less liquidity.
If we choose N = 100 we have 100 hubs. As before, the branch (1) is just a single sig and each hub requires 100 times less liquidity.
We can also use the previous idea of using multiple coinbase outputs. We can have 10 coinbase outputs, each with N = 100 hubs, so that would be 1000 hubs altogether, each requiring 1000 times less liquidity than the original proposal.
1000 is more of a demonstration to show how well this scheme scales. In reality it would run up against other bottlenecks like the bandwidth requirement for all those channel updates.
Re: Payment Channel Payouts: An Idea for Improving P2Pool Scalability
Maybe an obvious point, but the successful hashers (successful grinders) can create multiple coinbase outputs to use multiple hubs, reducing the liquidity requirement of each hub.
An example of coinbase transactions paying to three hubs instead of one:
Using N coinbase outputs reduces the liquidity requirements of a hub by a factor of 1/N.
So if you chose N = 5, today's block reward of around 12.5 btc means each hub only needs to provide liquidity for a reward of 2.5 btc.
Each output only costs about 20 more bytes to create (and about the same to spend because of the segwit discount).
This is an easy way to have many more people being able to run these p2pool hubs. Because of pareto's principle, reducing requirements by 1/N means the number of additional people who own that many bitcoins is much more than a factor of N. (Note that hubs need to own around 50 times the block reward, so more than just 2.5 btc)
An example of coinbase transactions paying to three hubs instead of one:
Code:
no inputs to coinbase ----> Payment to hub 1 (block_reward/3 btc)
Payment to hub 2 (block_reward/3 btc)
Payment to hub 3 (block_reward/3 btc)
Payment to hub 2 (block_reward/3 btc)
Payment to hub 3 (block_reward/3 btc)
Using N coinbase outputs reduces the liquidity requirements of a hub by a factor of 1/N.
So if you chose N = 5, today's block reward of around 12.5 btc means each hub only needs to provide liquidity for a reward of 2.5 btc.
Each output only costs about 20 more bytes to create (and about the same to spend because of the segwit discount).
This is an easy way to have many more people being able to run these p2pool hubs. Because of pareto's principle, reducing requirements by 1/N means the number of additional people who own that many bitcoins is much more than a factor of N. (Note that hubs need to own around 50 times the block reward, so more than just 2.5 btc)
⭐ Merited by xdrpx (1)
Beta version released: https://github.com/chris-belcher/electrum-personal-server
Includes all the features required for this to be used practically: Merkle proofs, deterministic wallets, bech32 addresses, SSL, Core's multi-wallet support.
I've been using this with my mainnet wallets and its very very lightweight. A small python script that's nothing compared to the full node it connects to.
Includes all the features required for this to be used practically: Merkle proofs, deterministic wallets, bech32 addresses, SSL, Core's multi-wallet support.
I've been using this with my mainnet wallets and its very very lightweight. A small python script that's nothing compared to the full node it connects to.
By my reading of Bitcoin Core's source code, the RPC call gettxoutproof won't work if pruning is enabled and if the relevant block has been deleted from disk.
I'm interested in this for Electrum Personal Server. Electrum wallet requests a merkle proof for each transaction otherwise it will display it as "Not Verified". For a non-pruned node this can be easily found with gettxoutproof and sent to Electrum, but if the node is pruned this often won't work.
What might be the best solution for my situation? The only idea I can come up with is to write code for Bitcoin Core that stores transaction merkle proofs in wallet.dat along with the rest of the transactions. Those proofs could then be accessed via the gettransaction RPC call.
I'm interested in this for Electrum Personal Server. Electrum wallet requests a merkle proof for each transaction otherwise it will display it as "Not Verified". For a non-pruned node this can be easily found with gettxoutproof and sent to Electrum, but if the node is pruned this often won't work.
What might be the best solution for my situation? The only idea I can come up with is to write code for Bitcoin Core that stores transaction merkle proofs in wallet.dat along with the rest of the transactions. Those proofs could then be accessed via the gettransaction RPC call.
Nice concept. I don't like storing the wallet on the server tho. Also in some ways wouldn't this reduce anonymity? Transactions correlate wallets with a certain IP address every time...
It's called "server" only because it implements the Electrum server protocol. In practice you'd run it on your own hardware, generally in the same place where your full node runs so the anonymity would not be harmed in this respect.
OK, but then you're back to leaking your IP address again.
I don't see how. If you connect to something on localhost you don't leak the IP of your Electrum wallet.
But the IP of the "private server" is the same as the IP of the wallet if you're connecting to localhost?!
Yes, but the electrum personal server is connected to a full node so you get the same privacy as if you were using Bitcoin-Qt. Bitcoin-Qt can have very strong privacy properties in terms of IP addresses.
Nice concept. I don't like storing the wallet on the server tho. Also in some ways wouldn't this reduce anonymity? Transactions correlate wallets with a certain IP address every time...
It's called "server" only because it implements the Electrum server protocol. In practice you'd run it on your own hardware, generally in the same place where your full node runs so the anonymity would not be harmed in this respect.
OK, but then you're back to leaking your IP address again.
I don't see how. If you connect to something on localhost you don't leak the IP of your Electrum wallet.
Re: Payment Channel Payouts: An Idea for Improving P2Pool Scalability
An interesting exchange with something familiar with mining in China
https://www.reddit.com/r/Bitcoin/comments/7wyq10/urgent_call_to_miners_mining_at_btccom_btccom_is/du4uoo1/?context=6
https://archive.is/Wt2FJ
So looks like a good p2pool selling point is fraud-resistance, since the pool cannot cheat the hashers out of their payments.
https://www.reddit.com/r/Bitcoin/comments/7wyq10/urgent_call_to_miners_mining_at_btccom_btccom_is/du4uoo1/?context=6
https://archive.is/Wt2FJ
So looks like a good p2pool selling point is fraud-resistance, since the pool cannot cheat the hashers out of their payments.
Nice concept. I don't like storing the wallet on the server tho. Also in some ways wouldn't this reduce anonymity? Transactions correlate wallets with a certain IP address every time...
It's called "server" only because it implements the Electrum server protocol. In practice you'd run it on your own hardware, generally in the same place where your full node runs so the anonymity would not be harmed in this respect.
Also from the README:
Electrum wallet must be configured to connect to the server. SSL must be disabled
Electrum wallet must be configured to connect to the server. SSL must be disabled
Pull requests welcome ¯\_(ツ)_/¯
All you have to do is read this page and figure out how to make the server socket be SSL. Since you're generally only connecting to localhost when using Electrum Personal Server, there's no privacy or security difference between SSL and non-SSL.
This is regarding the concerns raised in the first post:
- Electrum downloads block headers from multiple servers to prevent any one server from tricking you.
- Electrum downloads block headers from multiple servers to prevent any one server from tricking you.
This doesn't remove the need for trust, it only spreads it out. As the saying goes: don't trust, verify.
- What you're calling a gateway is actually a proxy. There is one built into tails called tor proxy which routes your electrum connections over the high anonymity tor network. The Electrum that comes with tails is pre-configured to use that. If you install an updated version on Tails you can configure it to use tor proxy too (i believe you have to use tor proxy on tails). Go to tools > network > connection tab, check ssl, choose socks 5 as the proxy, enter localhost as the host and 9050 as the port.
Tails with tor only stops your IP address being revealed, but the bitcoin addresses in your wallet are still linked together, and so the spying Electrum server can see still everything you do.
Using a full node solves this privacy break because it downloads the entire blockchain and scans it locally for your own addresses, so any external observer watching your connections can't tell which bitcoin addresses are yours.
⭐ Merited by AGD (2)
I just uploaded some code to a github repository (https://github.com/chris-belcher/electrum-personal-server) and sent this email to the bitcoin dev mailing list.
edit: link to email https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2018-February/015707.html
Subject: Electrum Personal Server alpha release
Electrum is a popular bitcoin wallet, but it is not a full node wallet as it synchronizes itself using third-party Electrum servers. The servers must be trusted to verify the rules of bitcoin, they can trick Electrum wallets into accepting fake bitcoin transactions which, for example, print infinite money. Bitcoin's security model requires that most economic activity is backed by full nodes. The Electrum servers must also be trusted with the user's privacy, as wallets send all their bitcoin addresses to the server. Spying on wallets is not much more complicated than simply grepping the server logs. Electrum wallets by default also connect to servers using their own IP address, linking it further to their revealed bitcoin addresses.
A way to avoid these problems is for users to run their own Electrum server and connect their wallets only to it. But this requires significant resource usage: the full unpruned blockchain, transaction index and an extra address index, as well as more RAM and CPU usage compared to just a full node. Servers are not well suited to being shut down and started up again, they are typically always online.
Electrum servers store a database of every bitcoin address ever used, which is inherently not scalable. This is resource-intensive and therefore pushes users towards centralized solutions. An alternative way would be to store only your own addresses and transactions.
Introducing Electrum Personal Server; an implementation of the Electrum server protocol which fulfills the specific need of using the Electrum UI with full node verification and privacy, but without the heavyweight server backend, for a single user. It allows the user to benefit from all of Bitcoin Core's resource-saving features like pruning, blocksonly and disabled txindex. All of Electrum's feature-richness like hardware wallet integration, multisignature wallets, offline signing, mnemonic recovery phrases and so on can still be used, but backed by the user's own full node.
An alpha version of Electrum Personal Server can be found on the repository: https://github.com/chris-belcher/electrum-personal-server
Before using, the wallet user must configure Electrum Personal Server with their master public key and those addresses are imported into Bitcoin Core as watch-only. If the wallet contains historical transactions then it must be rescanned. One of Electrum's motivating features is "instant on", which is therefore traded away when using Electrum Personal Server in return for full node verification and privacy. Although if a brand new empty wallet is created there is no need to rescan. A script like Electrum Personal Server is also well suited to use private transaction broadcasting tech like dandelion or broadcasting through tor.
Using Electrum with Electrum Personal Server is probably the most resource-efficient way right now to use a hardware wallet connected to your own full node. People who make use of Blockstream Satellite could use it to have an off-the-grid node connected to Electrum if that is their preferred wallet. In the situation of a traveller staying a cheap hostels, they could sync their node every couple of days to download recent blocks and use Electrum. Hopefully this software can be part of the plan to get full node wallets into the hands of as many people as possible.
The same kind of ideas could be applied to other lightweight wallets. For example a full nodes can run on smartphones with pruning and blocksonly, then a similar script would allow the user to connect their Samourai Wallet, Breadwallet or GreenAddress app to their own full node.
Further Reading:
* https://bitcointalk.org/index.php?topic=2664747.msg27179198
* https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-September/015030.html
* https://bitcointalk.org/index.php?topic=1634967.0;all
edit: link to email https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2018-February/015707.html
Subject: Electrum Personal Server alpha release
Electrum is a popular bitcoin wallet, but it is not a full node wallet as it synchronizes itself using third-party Electrum servers. The servers must be trusted to verify the rules of bitcoin, they can trick Electrum wallets into accepting fake bitcoin transactions which, for example, print infinite money. Bitcoin's security model requires that most economic activity is backed by full nodes. The Electrum servers must also be trusted with the user's privacy, as wallets send all their bitcoin addresses to the server. Spying on wallets is not much more complicated than simply grepping the server logs. Electrum wallets by default also connect to servers using their own IP address, linking it further to their revealed bitcoin addresses.
A way to avoid these problems is for users to run their own Electrum server and connect their wallets only to it. But this requires significant resource usage: the full unpruned blockchain, transaction index and an extra address index, as well as more RAM and CPU usage compared to just a full node. Servers are not well suited to being shut down and started up again, they are typically always online.
Electrum servers store a database of every bitcoin address ever used, which is inherently not scalable. This is resource-intensive and therefore pushes users towards centralized solutions. An alternative way would be to store only your own addresses and transactions.
Introducing Electrum Personal Server; an implementation of the Electrum server protocol which fulfills the specific need of using the Electrum UI with full node verification and privacy, but without the heavyweight server backend, for a single user. It allows the user to benefit from all of Bitcoin Core's resource-saving features like pruning, blocksonly and disabled txindex. All of Electrum's feature-richness like hardware wallet integration, multisignature wallets, offline signing, mnemonic recovery phrases and so on can still be used, but backed by the user's own full node.
An alpha version of Electrum Personal Server can be found on the repository: https://github.com/chris-belcher/electrum-personal-server
Before using, the wallet user must configure Electrum Personal Server with their master public key and those addresses are imported into Bitcoin Core as watch-only. If the wallet contains historical transactions then it must be rescanned. One of Electrum's motivating features is "instant on", which is therefore traded away when using Electrum Personal Server in return for full node verification and privacy. Although if a brand new empty wallet is created there is no need to rescan. A script like Electrum Personal Server is also well suited to use private transaction broadcasting tech like dandelion or broadcasting through tor.
Using Electrum with Electrum Personal Server is probably the most resource-efficient way right now to use a hardware wallet connected to your own full node. People who make use of Blockstream Satellite could use it to have an off-the-grid node connected to Electrum if that is their preferred wallet. In the situation of a traveller staying a cheap hostels, they could sync their node every couple of days to download recent blocks and use Electrum. Hopefully this software can be part of the plan to get full node wallets into the hands of as many people as possible.
The same kind of ideas could be applied to other lightweight wallets. For example a full nodes can run on smartphones with pruning and blocksonly, then a similar script would allow the user to connect their Samourai Wallet, Breadwallet or GreenAddress app to their own full node.
Further Reading:
* https://bitcointalk.org/index.php?topic=2664747.msg27179198
* https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-September/015030.html
* https://bitcointalk.org/index.php?topic=1634967.0;all
I was talking to shinobimonkey about this, he's been doing it for 12 months now.
He uses Samsung edge 6. Initial synchronization took him about a week with him leaving it on charger as much as possible. He sync'd at around block height 450000 (so about January 2017), and both blocksonly and pruning are enabled. He says he used a version of bitcoin core 0.14 or thereabouts. The ABCore GUI has an option to only allow sync'ing when on wifi and charging.
I'm actually impressed how near-possible it is to be running a practical smartphone full node today.
Smartphone full nodes will be important going ahead because smartphone usage has already outpaced desktop for some use cases (e.g. web browsing) and is only going up from here.
He uses Samsung edge 6. Initial synchronization took him about a week with him leaving it on charger as much as possible. He sync'd at around block height 450000 (so about January 2017), and both blocksonly and pruning are enabled. He says he used a version of bitcoin core 0.14 or thereabouts. The ABCore GUI has an option to only allow sync'ing when on wifi and charging.
I'm actually impressed how near-possible it is to be running a practical smartphone full node today.
Smartphone full nodes will be important going ahead because smartphone usage has already outpaced desktop for some use cases (e.g. web browsing) and is only going up from here.
I was reading this article https://www.coindesk.com/jonas-schnelli-wants-run-bitcoin-full-node/
In it Jonas Schnelli talks about running full nodes on smartphones.
I realize you could create a similar gateway script for other SPV wallets. For example you run a full node on your phone with a gateway, and then point your Breadwallet/Greenaddress/Samuraiwallet software to connect only to your own full node via gateway. This would be good in situations where people want to use those wallet UIs.
edit: Also these gateways can have private transaction broadcasting over tor built-in.
In it Jonas Schnelli talks about running full nodes on smartphones.
I realize you could create a similar gateway script for other SPV wallets. For example you run a full node on your phone with a gateway, and then point your Breadwallet/Greenaddress/Samuraiwallet software to connect only to your own full node via gateway. This would be good in situations where people want to use those wallet UIs.
edit: Also these gateways can have private transaction broadcasting over tor built-in.
Here is a conversation with nopara73, waxwing, aruk and I.
Quote
@belcher There is something I don't understand. Why not just run an Electrum server on your own machine and point the Electrum client to localhost?
ill write an FAQ with that since that question has come up a few times
I get it that pruning and stuff doesn't work like that.
the answer is because a normal electrum server needs lots of disk space and memory
yeah thats it
an regular electrum server has a database of every bitcoin address that was ever used
this gateway thing only has your own addresses, and they are stored in bitcoin core's wallet.dat
However instead of building a completely new system, for me it seems like the first reasonable approach would be try to tweak the existing system into being more space effective.
theres tweaking and then theres trying different approaches
storing every address ever used is inherently not scalable, storing only your own addresses is better for some things
the tradeoff is you have to -rescan if you import addresses that have historical transactions
I see, makes sense.
thats why this probably wont be added to electrum itself because it has "instant on" as one important value
this isnt "instant on", it trades away that in return for full node verification and less resource usage
Oh, there's something else in my mind.
What do you think of the future of on-chain transactions?
It seems like they are going to the roof.
And the situation will not get better.
they'll be limited
my view is basically the same as bitcoin cores/UASFs
block space is a scarce resource and we'll have to learn to use it efficently
This may comes with a trend of people who are using Bitcoin on-chain will also be easily able to afford to run a fullnode.
for privacy specifically i wrote this https://www.reddit.com/r/joinmarket/comments/7m422w/some_updates_on_joinmarketclientserver/drtzm84/
privacy will have to happen off-chain, because buying lots of block space with coinjoin/whatever will be expensive
reading it again now, i should put chaumian ecash servers as another off-chain tech and link your blog post
Recently I was wondering if building a nice UI on top of fully validating Core is a strategically smarter decision than trying to tweak with light clients.
yes i think so, i really like electrum's UI so my gateway idea is about having both electrum's nice UI and core's full verification
in your bitcointalk reply you say "Building a nice UI would be less of an effort then building this gateway." which is... not true... building a nice UI means re-doing all the work that the electrum devs have done
yes agreed, it's not really even quite "nice UI" that electrum client has got so right, it's feature-richness imo.
anyway if its okay with you id like to copypaste this conversation into that bitcointalk thread
feel free to repeat anything i said.
@belcher My point was building a UI is just putting one thing on top of another, while building a gateway requires actual engineering and logic. Maybe building a UI would be more work, but it would be an easier work, where you don't have to be anxious about if your algo fails, people can lose money.
@nopara73 that's true, but he doesn't want to change the UI, he wants to change how easily you can replicate the electrum server functionality backend as a single user with one node
you can do it now but it's too expensive (txindex and so on)
Ok, but that's just a personal engineering boner, not an actual, "what's the most effective way to fix that problem I have: decision:)
you think so? i don't. if you scroll up you'll see i made the same point as you about reusing existing arch; but he intends to create a far lighter-weight version of it, which would address a specific need: to use electrum UI without a heavyweight server backend, for a single user.
Indeed. I didn't understand what you were talking about, because when I read your comments I didn't yet read the bitcointalk post.
It would actually be very useful as using electrum server with your full node is basically the only way to validate your hardware wallet transactions with a full node
I wonder why this is not a more wanted feature
aruk yes that is one big motivation, i realized we talk up hardware wallets as a great way to store privkeys, but bitcoin security also requires using a full node wallet and hardware wallets dont work well with those (yet)
for engineering vs UI, i dont like UI that much (personally)
it does but it's extremely cumbersome
i actually liked reading cs textbooks with all those graphs and heaps and tree diagrams
creating UI i mean
using it is obviously nice
only thing is, like with JM, if you go with a non-indexed approach, you have problems in recovery.
yep, tradeoffs :\
its required for scalability i think, transactions come in blocks, you have to scan them
@waxwing My point is you get the same result if you replicate the Electrum GUI for, Core, or let's say @jonasschnellis fullspv PR here: https://github.com/bitcoin/bitcoin/pull/9483 then it's an easier way to get to where he wants to get, than writing the gateway.
you could make a database of every address every used... but that will be big and probably centralized because of that (i.e. people will use bc.i's API instead of scanning their own blocks)
nopara73 jonasschnellis PR doesnt support (for example) mnemonic phrases
or any other of electrum's nice features
yeah it isn't just GUI, like i said above, it's feature-richness.
yes, UI is the wrong word
HW wallet integration mentioned above is a very good point too.
several other things, but i don't want to be unfair to Core's GUI since i never used it :laughing:
features are a part of UI i guess... /me looks up how "UI" is actually defined
What? Why doesn't it? I support it in HiddenWallet. It's just the user must specify a date when the wallet was created.
At recovery
if you want consensus to be enforced, you need people validating high value transactions. Nobody is doing it because they all use a hw wallet that does not depend on a local node
Core's GUI doesnt support lots of stuff, i think most core developers are interested in the super-important scalability stuff
there's a huge amount of work gone into scalability in bitcoin core, without which bitcoin would likely die
IMO the best application in Core is bitcoind not bitcoin-qt
the way i run things is an old always-on laptop with bitcoind running (and other stuff like joinmarket and irc)
ill write an FAQ with that since that question has come up a few times
I get it that pruning and stuff doesn't work like that.
the answer is because a normal electrum server needs lots of disk space and memory
yeah thats it
an regular electrum server has a database of every bitcoin address that was ever used
this gateway thing only has your own addresses, and they are stored in bitcoin core's wallet.dat
However instead of building a completely new system, for me it seems like the first reasonable approach would be try to tweak the existing system into being more space effective.
theres tweaking and then theres trying different approaches
storing every address ever used is inherently not scalable, storing only your own addresses is better for some things
the tradeoff is you have to -rescan if you import addresses that have historical transactions
I see, makes sense.
thats why this probably wont be added to electrum itself because it has "instant on" as one important value
this isnt "instant on", it trades away that in return for full node verification and less resource usage
Oh, there's something else in my mind.
What do you think of the future of on-chain transactions?
It seems like they are going to the roof.
And the situation will not get better.
they'll be limited
my view is basically the same as bitcoin cores/UASFs
block space is a scarce resource and we'll have to learn to use it efficently
This may comes with a trend of people who are using Bitcoin on-chain will also be easily able to afford to run a fullnode.
for privacy specifically i wrote this https://www.reddit.com/r/joinmarket/comments/7m422w/some_updates_on_joinmarketclientserver/drtzm84/
privacy will have to happen off-chain, because buying lots of block space with coinjoin/whatever will be expensive
reading it again now, i should put chaumian ecash servers as another off-chain tech and link your blog post
Recently I was wondering if building a nice UI on top of fully validating Core is a strategically smarter decision than trying to tweak with light clients.
yes i think so, i really like electrum's UI so my gateway idea is about having both electrum's nice UI and core's full verification
in your bitcointalk reply you say "Building a nice UI would be less of an effort then building this gateway." which is... not true... building a nice UI means re-doing all the work that the electrum devs have done
yes agreed, it's not really even quite "nice UI" that electrum client has got so right, it's feature-richness imo.
anyway if its okay with you id like to copypaste this conversation into that bitcointalk thread
feel free to repeat anything i said.
@belcher My point was building a UI is just putting one thing on top of another, while building a gateway requires actual engineering and logic. Maybe building a UI would be more work, but it would be an easier work, where you don't have to be anxious about if your algo fails, people can lose money.
@nopara73 that's true, but he doesn't want to change the UI, he wants to change how easily you can replicate the electrum server functionality backend as a single user with one node
you can do it now but it's too expensive (txindex and so on)
Ok, but that's just a personal engineering boner, not an actual, "what's the most effective way to fix that problem I have: decision:)
you think so? i don't. if you scroll up you'll see i made the same point as you about reusing existing arch; but he intends to create a far lighter-weight version of it, which would address a specific need: to use electrum UI without a heavyweight server backend, for a single user.
Indeed. I didn't understand what you were talking about, because when I read your comments I didn't yet read the bitcointalk post.
It would actually be very useful as using electrum server with your full node is basically the only way to validate your hardware wallet transactions with a full node
I wonder why this is not a more wanted feature
aruk yes that is one big motivation, i realized we talk up hardware wallets as a great way to store privkeys, but bitcoin security also requires using a full node wallet and hardware wallets dont work well with those (yet)
for engineering vs UI, i dont like UI that much (personally)
it does but it's extremely cumbersome
i actually liked reading cs textbooks with all those graphs and heaps and tree diagrams
creating UI i mean
using it is obviously niceonly thing is, like with JM, if you go with a non-indexed approach, you have problems in recovery.
yep, tradeoffs :\
its required for scalability i think, transactions come in blocks, you have to scan them
@waxwing My point is you get the same result if you replicate the Electrum GUI for, Core, or let's say @jonasschnellis fullspv PR here: https://github.com/bitcoin/bitcoin/pull/9483 then it's an easier way to get to where he wants to get, than writing the gateway.
you could make a database of every address every used... but that will be big and probably centralized because of that (i.e. people will use bc.i's API instead of scanning their own blocks)
nopara73 jonasschnellis PR doesnt support (for example) mnemonic phrases
or any other of electrum's nice features
yeah it isn't just GUI, like i said above, it's feature-richness.
yes, UI is the wrong word
HW wallet integration mentioned above is a very good point too.
several other things, but i don't want to be unfair to Core's GUI since i never used it :laughing:
features are a part of UI i guess... /me looks up how "UI" is actually defined
What? Why doesn't it? I support it in HiddenWallet. It's just the user must specify a date when the wallet was created.
At recovery
if you want consensus to be enforced, you need people validating high value transactions. Nobody is doing it because they all use a hw wallet that does not depend on a local node
Core's GUI doesnt support lots of stuff, i think most core developers are interested in the super-important scalability stuff
there's a huge amount of work gone into scalability in bitcoin core, without which bitcoin would likely die
IMO the best application in Core is bitcoind not bitcoin-qt
the way i run things is an old always-on laptop with bitcoind running (and other stuff like joinmarket and irc)
Electrum uses Electrum servers which can spy on or lie to wallet users. Electrum wallets may be tricked into accepting transactions of BitcoinXT or some other hostile hard fork proposal. And the servers have a list of all the user's bitcoin addresses which is harmful to privacy. Right now Electrum comes packaged with Tails OS which has the tagline "Privacy for anyone anywhere", which is simply not true when using Electrum with default servers.
Another way to synchronize a wallet would be to download full blocks and/or point the wallet to your own full node. One of Electrum's values is "instant on" so those methods will probably never be added to Electrum itself.
Probably a better way would be to create a gateway, a script that behaves like an Electrum server but which obtains bitcoin network information in another way. To use it you would just run the gateway script and point your Electrum wallet to localhost. The script would be separate from the Electrum software itself so Electrum's raison d'etre which includes "instant on" would be preserved.
This is basically a proposal for an improved lightweightness of Electrum server. The goal is to keep all of Electrum's feature-richness of hardware wallet integration, multisignature, offline signing, coin control, etc but with better privacy, full node verification and minimal resource requirement compared to a full server.
Here are some gateway script ideas:
Type 1
The gateway script adds watch-only addresses to Bitcoin Core's wallet via the RPC protocol. Updates to the wallet.dat are transmitted to Electrum via the gateway.
Features: Full verification, security and privacy from a full node. Works with pruning and blocksonly. Reasonably simple to code.
Downsides: Adding wallets with historical transaction being added will require rescanning. Users must configure the gateway script with their Electrum master public keys before using.
Type 1 would be combining Bitcoin Core with Electrum. All of Bitcoin's configuration tuning like -blocksonly and -pruning can be used and txindex can be switched off. It would provide a way to use Electrum wallet with as low resource usage as possible while still being a full node. It would also be reasonably simple to code, only requiring Electrum's protocol and Bitcoin's RPC to be implemented and have them connected together.
I've heard a story of a tourist travelling through rural South America with a laptop, every couple of days at the cheap hostels they stayed they would run Bitcoin with -blocksonly to catch up. The type 1 gateway script would allow them to use Electrum if that was their preferred wallet.
Regarding rescanning. The user should have all their Electrum wallets imported into Bitcoin Core's wallet.dat file, so then they can switch between them without rescanning. If a brand new empty wallet with no transactions is created, there is also no need to rescan. The only time a rescan is needed is if new wallets are added which have historical transactions.
Type 2
The gateway script connects to the bitcoin p2p network and downloads full blocks (possibly with committed block bloom filters).
Features: Excellent privacy, almost as good as full node. SPV security just like Electrum today. Doesn't require any setup.
Downsides: Uses (much) more initial bandwidth than an Electrum server. Slower startup. Fairly complicated to code. Requires using new technology (committed bloom filters) before it can be practical.
Type 2 would be trading off bandwidth and speed for privacy. The wallet downloads some full blocks which takes longer, but in return it gets much better privacy. This script could be added to Tails OS as a pre-installed pre-configured script. Note that scanning blocks only needs to start from the wallet creation date, so if the user creates their wallet right now then the gateway script won't need to scan any old blocks.
Thoughts?
Another way to synchronize a wallet would be to download full blocks and/or point the wallet to your own full node. One of Electrum's values is "instant on" so those methods will probably never be added to Electrum itself.
Probably a better way would be to create a gateway, a script that behaves like an Electrum server but which obtains bitcoin network information in another way. To use it you would just run the gateway script and point your Electrum wallet to localhost. The script would be separate from the Electrum software itself so Electrum's raison d'etre which includes "instant on" would be preserved.
This is basically a proposal for an improved lightweightness of Electrum server. The goal is to keep all of Electrum's feature-richness of hardware wallet integration, multisignature, offline signing, coin control, etc but with better privacy, full node verification and minimal resource requirement compared to a full server.
Here are some gateway script ideas:
Type 1
The gateway script adds watch-only addresses to Bitcoin Core's wallet via the RPC protocol. Updates to the wallet.dat are transmitted to Electrum via the gateway.
Features: Full verification, security and privacy from a full node. Works with pruning and blocksonly. Reasonably simple to code.
Downsides: Adding wallets with historical transaction being added will require rescanning. Users must configure the gateway script with their Electrum master public keys before using.
Type 1 would be combining Bitcoin Core with Electrum. All of Bitcoin's configuration tuning like -blocksonly and -pruning can be used and txindex can be switched off. It would provide a way to use Electrum wallet with as low resource usage as possible while still being a full node. It would also be reasonably simple to code, only requiring Electrum's protocol and Bitcoin's RPC to be implemented and have them connected together.
I've heard a story of a tourist travelling through rural South America with a laptop, every couple of days at the cheap hostels they stayed they would run Bitcoin with -blocksonly to catch up. The type 1 gateway script would allow them to use Electrum if that was their preferred wallet.
Regarding rescanning. The user should have all their Electrum wallets imported into Bitcoin Core's wallet.dat file, so then they can switch between them without rescanning. If a brand new empty wallet with no transactions is created, there is also no need to rescan. The only time a rescan is needed is if new wallets are added which have historical transactions.
Type 2
The gateway script connects to the bitcoin p2p network and downloads full blocks (possibly with committed block bloom filters).
Features: Excellent privacy, almost as good as full node. SPV security just like Electrum today. Doesn't require any setup.
Downsides: Uses (much) more initial bandwidth than an Electrum server. Slower startup. Fairly complicated to code. Requires using new technology (committed bloom filters) before it can be practical.
Type 2 would be trading off bandwidth and speed for privacy. The wallet downloads some full blocks which takes longer, but in return it gets much better privacy. This script could be added to Tails OS as a pre-installed pre-configured script. Note that scanning blocks only needs to start from the wallet creation date, so if the user creates their wallet right now then the gateway script won't need to scan any old blocks.
Thoughts?
Re: [ANN] Joinmarket - Coinjoin that people will actually use
Estimated fee per kB greater than absurd value: 350000, quitting.
Without getting any more information, it looks like this is the problem.
Recently demand for block space has been very high, so when your joinmarket bot obtains fee estimations it returned a value higher than 350 sat/b (350000) which joinmarket interpreted as an absurd value.
Try opening `joinmarket.cfg` and changing `absurd_fee_per_kb = 350000` to another higher value.
Can't do it. That breaks the decentralized concept and ruins the system. Yes, there are many people that have at least one issue and lose coins within their crypto lifetime. This can only be controlled by their knowledge. Some sites provide things like escrow, but that is not within the coin code and can never be.
It doesn't break any decentralized concept.
Re: [ANN] Joinmarket - Coinjoin that people will actually use
Looks like electrum uses the WIP version byte to choose which kind of address it generates https://github.com/spesmilo/electrum/blob/3.0.1/RELEASE-NOTES#L46-L55
1st August 2017 - "UASF day" BIP148 trigger date for activating UASF for segwit.
9th August 2017 - Segwit soft fork locks-in, the point of no return when we became sure that bitcoin will have segwit.
23rd August 2017 - Segwit activates and is available for use by anyone.
8th November 2017 - 2X hard fork attempt is abandoned. https://twitter.com/pavolrusnak/status/928386661528035329
9th August 2017 - Segwit soft fork locks-in, the point of no return when we became sure that bitcoin will have segwit.
23rd August 2017 - Segwit activates and is available for use by anyone.
8th November 2017 - 2X hard fork attempt is abandoned. https://twitter.com/pavolrusnak/status/928386661528035329