Hey all,

I'm going to close this topic now in line with the foundations' desire to have forum communication over at mastercointalk.org... See you over there!

Thanks
Zathras

Hey guys,

0.4 is up.  This includes SP support (display & send).

I'd like to take a moment to remind everyone that this is experimental alpha testing software and all use is at your own risk - please remember this if you're considering using it with anything other than small testing amounts.

Also, while I have you

0.4 will be the last Masterchest release of the wallet.  As of now the project is being handed over to the foundation and they will own the development and direction of future desktop wallet development.  Since I know the tendency for people to interpret differently is very prevalent in the Crypto space, I'd like to make a few things very clear as follows:

* I am not stopping development, nor am I stepping down in my role as RBB Developer for the foundation
* 0.4 is the last "Masterchest" release, but that does not mean last release period.  The foundation will of course be continuing the development of wallet software - perhaps my .NET wallet, perhaps something else.  Desktop wallet direction is a hotly discussed topic at the moment, but one thing we all agree on is a need for a consolidated direction - moving the wallet project to the foundation is a logical step in pursuit of this
* I will be continuing work on other Masterchest projects, most notably the Masterchest.info block explorer will continue to be actively developed
* I will be contributing to a consolidated wallet development effort, though not leading it, regardless of the direction chosen for development

With those points out the way (I would hate for this to be construed as negative), I just wanted to add a personal note; this project was launched as an idea, but an idea that needed software to turn it into something tangible and Masterchest was a rapid prototype to bridge that gap.  It's been a wonderful (if tiring!) journey but as we grow and evolve into something more mature, more organized and more focused the 'one-man-band' approach that is Zathras & Masterchest is of course not a long term scalable prospect (or good for the project) - but I'm extremely proud to have helped the project along in my own little way & look forward to contributing to a consolidated & shared development effort as we charge ever further ahead!

Download

Thanks
Zathras

Hey all,

0025 is available for download, corrects an issue where action byte is not enforced for case 0x02 where no previous sell exists (should be invalidated) and applies formula fix for rounding on purchase amounts.

Please update.

Thanks
Zathras

Hi, yep sorry about that, the previous fix on expiry was too aggressive and caused pending buys already paid to appear expired (self correcting one the payment was confirmed), that behaviour should now be fixed.

Thanks
Zathras

Hey all,

Please note 0024 is available for download, corrects an issue where accept offers are displayed as expired while payment is pending.

Thanks
Zathras

Hey,

I ran this through virustotal and wasn't able to see any issues - I actually recommend putting an exception in for any resident shield or network proxy, as if it proxies all the network requests (RPC) to bitcoind/qt it slows things down by >50%.

Thanks
Zathras

Build 0023 is available, please update.  This fixes a critical bug that flags payments for DEx purchases in the block immediately following expiry as valid (incorrectly).

If you would like further explanation please see below.

Binary via Github: Download

Thanks
Zathras

In a Master Protocol distributed exchange transaction the seller decides on a 'payment window' - a length of time (in blocks) for the buyer to make payment after his acceptance has been confirmed in the blockchain.  After this payment window, the acceptance expires.

If for example the seller has specified a time limit of 10 blocks, and the buyer accepts the sell offer in block 300000, the buyer then has until block 300010 to make payment.

A bug in the way Masterchest processes expiration allowed these acceptances to remain pending for the block immediately following expiry in certain circumstances, thus potentially allowing a DEx payment to incorrectly be seen as valid if it was exactly one block late.

This bug was discovered via our consensus systems noting that Masterchest had deviated from the reference this morning (Masterchain.info - unaffected).  Masterchest softwares (wallet and block explorer) incorrectly validated the purchase accepted in https://masterchest.info/lookuptx.aspx?txid=0d2f2f864a51938721a17963aa63925da4cd67ab1072d9ae339c3c0073b1308c. 

This bug has now been corrected and all Masterchest users are strongly encouraged to update their wallet software.

OK thanks - just to double check are you fully re-indexed and fully synced (bitcoin wise)?  It looks like it from the screenshot, but bitcoin will throw a 500 if it can't find the transaction or block we're searching for.

The next library build will have some additional information provided when bitcoin returns internal server errors.

For an immediate fix on another topic, J.R. pointed me to a bug with empty history - was fixed in my smart property experimental branch but still buggy in master, so have committed that as an urgent fix.  If you get an exception because it's a fresh wallet or you've never done any Mastercoin transactions, please pull the download again for the fix.

Thanks for the info.  The next version will have additional debugging in failed RPC calls.  FYI the wallet is disposable - have you tried deleting the entire (Masterchest) wallet folder and pulling the download again?

Hey guys, sorry haven't been around much - head down coding

g0re79 can you post the output of 'getinfo' from your bitcoin installation?

Thanks
Zathras

I notice you're running build 0019, try the latest (0021) it has retry capability in the RPC calls if your bitcoin installation doesn't respond on the first try.  It's doubtful this will solve your issue (as it's an internal server error not connection error) but always good to be running the latest build.

Also, could you post the output on the debug panel?  

What antivirus platform are you running?  I've seen a few interfere with comms to RPC.  Any firewalling on the interface for bitcoin RPC? Is bitcoin fairly vanilla - any customization?  There are still some things that Masterchest won't support, for example force using IPv6 instead of IPv4 in bitcoin-core may cause problems (if you don't know what that is then don't worry you won't be using it).

Once we see which step it's up to in the debug panel perhaps I can give you a build with extra debugging at that step to shep some light on what's going on.  We do have users on Win8 x64 (unsure if IE10) using the wallet successfully, so I'm not sure if it's something specific to your local environment.

Thanks
Zathras

Hehe I wasted about an hour rotating that original sync icon a bunch of times to make an animation but the result was pretty poor - I think the new (properly generated) one looks much better - thanks for the ideas

I can change some of the defaults around, I think there is an issue for that already.

I could always grep the logs for the number of times that page is hit if needed sure.

Doing some sort of ticketing is great if a reasonable turnaround can be achieved - the difficulty with things today is people want their answers (now! now! now!) and even say 8 hours which used to be a good customer service turnaround is seen as slow (not that many in the crypto world deliver service even close to same day though!).

I think a decent FAQ needs to be added to that support landing page at minimum though.

EDIT: Perhaps that's expecting a bit much (8 hrs) - this is a very young industry after all - too much time in the day job negotiating SLAs with helpdesk

Build 0021 is up.

Minor bugfixes and some UI improvements (thanks for all the suggestions!).

Thanks
Zathras

Legend DexX - added.

Thanks
Zathras

EDIT: Notes are extremely useful, thanks

On the 'ugly' scrollbars - couldn't agree more however since this is WinForms and not WPF repainting scrollbars is very scrappy/hacky I think - if you have any suggestions here I'm open to them absolutely.

Thanks - have been wanting to get this out (but crazy couple of days as you know!) because I knew there was some low hanging fruit for optimization (+little indexing sprinkled here and there helps a lot!).

Ah yes, been meaning to simulate that (setting a locale in a VM that has , instead of . so I can replicate myself & test).  Remind me again does it also do that on the currencies panel?

Hmm, we enumerate addresses via multiple RPC methods (because contrary to the bitcoin-core doco, listaddresses does not return all addresses - for example change) but then we check every one via the validateaddress RPC call, using the ismine:true condition before adding to the address stack.  Could you do me a favour?  In your bitcoin install run the command validateaddress address on one of your MSC addresses while in your other wallet.  I can't see any way that bitcoin would return anything other than ismine:false for that but good to check.

I'm also trying to think of other ways this could occur.

Thanks as always for testing
Zathras

P.S.  You can interchange wallet.dat as much as you like (as long as the wallet isn't running of course) - wallet.sdf holds no unique information (otherwise we'd need to encrypt it, back it up etc).  Technically calling Masterchest a wallet is a bit of a misnomer, bitcoind/qt is the wallet, Masterchest just overlays an extra layer of functionality over the top (thus allowing me to conveniently inherit the security of the bitcoin-core wallet )

FYI:

Hey guys,

Alpha 0.3C is up.  OP updated.

This update focuses on:
* Major improvements in processing performance - parsing is about the same but my benchmarks show 200%+ improvement in processing - thus makes an up to date client much faster to use
* Hash based chain re-org protection (upgraded from always roll-back 5 blocks to storing hashes for blocks we process DB local and then comparing with bitcoind/qt when we start scanning)
* Some UI improvements, bugfixes

Thanks
Zathras

Well the thing is bitcoin uses heuristics to intelligently select coins to use intentionally to minimize the transaction weight on the network etc.  Masterchest does none of this and just follows the rules whether that results in an optimized transaction or not.  

Like a car from 50 years ago, it still gets from A to B but it could be more efficient.

I think the best way to come at it is to just go straight for proper adaptive transaction crafting and make Masterchest smart enough to handle some context - eg a couple examples from one of my git comments:

Can I craft this transaction in a way that leaves unspent inputs available for further Master Protocol transactions from this address same block?
Can I craft this transaction in a way that minimizes the fees payable by the user?
Can I craft this transaction in a way that consolidates some of the small reference outputs (.00006 etc)?

And so on...  After that we move from MP to Bitcoin considerations:

Can I craft this transaction in a way that minimizes transaction size on the network?

Etc Etc

Appreciate the offer but I think I'll be OK - after looking at this a bit more I actually don't think it's that cumbersome to look at increasing fee after initial transaction creation & signing - even if the fee does need to be increased (and the total input doesn't cover it), that's at worst two extra >dust inputs which would be about ~300 bytes so not enough to trigger another fee increase - that nullifies my concerns that we'd get into a loop adding inputs to cover fees which grows the tx to require more inputs thus more fees, which grows the tx and rinse repeat.  Worst case is we have to go back and change things once and resign, which isn't so bad.

Thanks
Zathras

It's an interesting topic - receive lots of MP messages and you'll have lots of small ~0.00006 inputs - redeemimg them at the same time means a bigger transaction, thus requires more fees.

I actually think it's mostly about us devs being smart & creative about the way we craft our transactions to make sure we're spending these 0.00006 inputs where possible together with inputs from other transactions but at the same time not throwing around silly transactions with 50 tiny inputs.

That's actually a really interesting problem - especially in relation to these fractional transactions.  We build the transaction before we can see its total size, but by then we've already selected the inputs.  As we're talking such tiny amounts (often inputs of 0.00006) increasing the fee by 0.0001 likely means we have to add another/more inputs to cover the increased fee, but that has now changed the size of our transaction again so we may need to yet again go back and add more inputs & more fees and so on.  Fun game

In all seriousness though I do agree with DexX - changes to payments make me nervous but it should be safe.  I've pushed up a shortcut fix which grows the fee during transaction build using simple (very conservative 200 bytes per input) guestimation based on total number of inputs at that point in the transaction build.  Should take care of any potential risk for now until I can be a bit more classy with input selection


Note this only applies to DEx payment transactions, all other transaction types Masterchest selects a single input only to cover fees.

Thanks
Zathras

This is not so much a problem of the Master Protocol, but more one of how bitcoin works.  If I may take a moment to explain

Miners 'prioritize' transactions.  This means when your transaction is relayed to a given mining client it adds the transaction to its memory pool and assigns a priority and fee.  If the priority/fee is high enough, hopefully the transaction will be included in the next block.

Now - the interesting bit is in how this transaction priority is calculated by bitcoin core, as follows:

priority = sum(input_value_in_base_units * input_age)/size_in_bytes

Thus we can see the smaller and younger the inputs, the lower the transaction priority becomes.  

If we specifically relate this to your transaction, we can see that the inputs were all very small.  Additionally because there are a number of them, the size is actually over 1KB.  Thus we have a small input value and a (relatively) large byte size for the transaction.  If you pop those numbers into the formula above you'll see this provides for a very low priority transaction.

The good news (and this is bitcoin-wide) is that in laymans terms - the more you send, the higher your transaction priority.  Thus this may be an issue for low value fractional transactions and I'll do some further investigation and discuss with the team - but essentially as the value of the DEx purchase increases, the risk of the issue you describe occurring decreases.

I have also been doing some thinking on input selection, but that would apply only to the Masterchest implementation when I get there.  Currently I select smallest input for risk minimization (avoid losing a big input if something goes wrong creating the transaction).  On the todo list is adaptive transaction encoding in the library, so rather than 'dumb' input selection we look at input values, ages and eventual transaction size to determine priority before it's broadcast & if this would result in a LP transaction that is likely to take a while to get mined, re-encode it if possible for a higher priority, else send it with a higher fee.  That's a complex undertaking though so will take a little time (we always have to be careful when coding input selection and amounts as that's where the biggest potential to lose bitcoins sits IMO).

Thanks!
Zathras

EDIT: Also realized I wasn't explicit in stating it's of course also feasible to simply discourage (or disallow) fractional purchases of for example 0.000X BTC after a specific block height in the protocol itself too - the Master Protocol DEx has never been intended for high frequency or micro trading (and these micro trades cost more in fees than the trade itself).