Quote from: kano on November 16, 2016, 11:35:22 PM

Do you have numbers to compare - coz that's really the point here.
I of course have the old compression numbers for every block since the beginning of time since I've used your relay Tongue

If the new one is better compression (= faster block transmission) then you can say that if you have numbers

First of all, you're making a common mistake here - less data != faster block transmission when you're talking about things in the 1-20k range. The extra cost of a packet isnt a ton (loss is not hugely likely if you're on a decent connection until you start talking about 100k+ data) and the extra transmission time of a packet is virtually nothing - so an additional 20ms in receive->ProcessNewBlock time is often worth it. With stuff like https://github.com/bitcoin/bitcoin/pull/9125 this is very realistic.

Quote from: kano on November 16, 2016, 11:35:22 PM

Post a run of any 100 blocks and I'll get the same blocks and see (like my 50 block post above) - that's really what I want to see.

I dont have tons of numbers handy, but if you look at a bitcoind with -debug running that receives blocks not over the RN, then you can easily see compact block reconstruction info (eg blocktxn and cmpctblock message sizes). In my recent log for blocks received over compact blocks most blocks took between 12k and 17k for the blocktxn, and few needed a blocktxn roundtrip (printed in log as having received a 33-byte blocktxn, which isnt a real message on the wire).

Quote from: kano on November 16, 2016, 11:35:22 PM

Edit2: there's also another question in the interface design - what's the comparison of the 2 for back and forward communication?
e.g. comparing the two (I don't know what that are)

As noted in the previous post, the old RN protocol is one-way as far as blocks are concerned...it makes strong assumptions about knowledge of the other side's mempool (requires you to store the last N txn sent to you by the server), and can thus just send you the compressed block and knows you can reconstruct it. The Compact Blocks protocol isnt so nice, and sometimes requires a round-trip to request missing transactions, though being able to send the cmpctblock message 5-10ms sooner and saving 10-20ms on the receiving side can almost make up for that if you're (reasonably) close to one of the RN servers (eg AWS us-east or us-west should meet that criteria easily). Hopefully further optimizations to the compact block implementation (that the protocol designed for) in 0.14 will will make the likelyhood of round-trips much lower.

Quote from: kano on November 16, 2016, 11:35:22 PM

My main concern is that bitcoind sux.
It's hopeless at keeping connections and has too much overhead.
The relay seemed to be able to keep the connection for a long time (I have counters that display in a different colour when it loses/regains the connections) so I've seen how reliable that can be.
(I don't "watch" the old relay log, I instead have a script run a simple analysis of the last 9999 lines in the log every 10 seconds)
I suspect that the relay was pretty minimal in back and forth communication and data size vs bitcoind.

Yea, this is something I thought about a lot when considering the protocol change, actually this more than the latency concerns. While I dont know why it would be bad at keeping connections, aside from not reconnecting agressively enough, the agressive reconnection stuff that I have in the old RN is super nice since I can pretty freely restart the servers, etc. As for protocol-chatty-ness, I think they're not that far off, bitcoind will send more transactions, but thats a good thing - more transactions to match the compact block against

.