Not sure exactly what you're trying to do here...As o_solo_miner pointed out, to use the public relay network for receiving-blocks-quickly, you should simply run unmodified Bitcoin Core 14/15 and addnode to one of the FIBRE nodes (see http://bitcoinfibre.org/public-network.html).

If you wish to run your own FIBRE network to propagate blocks quickly between your own nodes, note that the udpport syntax (as pointed out in the --help output) is udpport=port,group,bw where port is a port number, group is a monotonically-increasing group id (ie it should be 0, the second port should use 1, etc) and bw is a bandwidth to use for sending to that group, in Mbps.

The udpnode stuff is not exposed on the public FIBRE instances because I dont significantly trust that there isnt some bug in the implementation somewhere which could cause the nodes to crash with data sent corrupted in some way. The other option (the fibrenetworkclient thing from my RelayNode github) exists only to make sure connections stay up reliably. See two posts up (where we talked about https://github.com/bitcoin/bitcoin/pull/9319), which will make -addnode a much more reliable "keep this connection open" option in Bitcoin Core 0.14. fibrenetworkclient shouldn't make any performance difference at all.

This has nothing to do with getblock latency. This has to do with the time until you see a block. p2pool has no idea how long it took from when someone else found a block to when you received it, and this is the part that relay networks work to solve.

See https://github.com/bitcoin/bitcoin/pull/9319.

For those using the new FIBRE-based Relay Network, I just added the following paragraph to the public-network page:

"Note that, because bitcoind treats the -addnode argument as an extra seednode it may use, and not as a connection which it should maintain, it is recommended that you use an external daemon to keep the connection reliable. There is a simple connect-two-bitcoinds-together client available in the Relay Network Client Source Tree (called fibrenetworkclient after you run make) which takes, as arguments, two bitcoind instances, and connects them to each other, reliably reconnecting if one goes down."

First of all, you're making a common mistake here - less data != faster block transmission when you're talking about things in the 1-20k range. The extra cost of a packet isnt a ton (loss is not hugely likely if you're on a decent connection until you start talking about 100k+ data) and the extra transmission time of a packet is virtually nothing - so an additional 20ms in receive->ProcessNewBlock time is often worth it. With stuff like https://github.com/bitcoin/bitcoin/pull/9125 this is very realistic.

I dont have tons of numbers handy, but if you look at a bitcoind with -debug running that receives blocks not over the RN, then you can easily see compact block reconstruction info (eg blocktxn and cmpctblock message sizes). In my recent log for blocks received over compact blocks most blocks took between 12k and 17k for the blocktxn, and few needed a blocktxn roundtrip (printed in log as having received a 33-byte blocktxn, which isnt a real message on the wire).

As noted in the previous post, the old RN protocol is one-way as far as blocks are concerned...it makes strong assumptions about knowledge of the other side's mempool (requires you to store the last N txn sent to you by the server), and can thus just send you the compressed block and knows you can reconstruct it. The Compact Blocks protocol isnt so nice, and sometimes requires a round-trip to request missing transactions, though being able to send the cmpctblock message 5-10ms sooner and saving 10-20ms on the receiving side can almost make up for that if you're (reasonably) close to one of the RN servers (eg AWS us-east or us-west should meet that criteria easily). Hopefully further optimizations to the compact block implementation (that the protocol designed for) in 0.14 will will make the likelyhood of round-trips much lower.

Yea, this is something I thought about a lot when considering the protocol change, actually this more than the latency concerns. While I dont know why it would be bad at keeping connections, aside from not reconnecting agressively enough, the agressive reconnection stuff that I have in the old RN is super nice since I can pretty freely restart the servers, etc. As for protocol-chatty-ness, I think they're not that far off, bitcoind will send more transactions, but thats a good thing - more transactions to match the compact block against .

Not sure what "excessive burden" you're referring to here - replacing one P2P connection with another shouldn't change the load on your bitcoind, even if one is remote...If you really want to split hairs you should see less load on your bitcoind because the message hashing for an entire block no longer happens.

Not quite...the old relay network protocol is bandwidth-limited to only send out a certain number of transactions per second (I think its like 5Mbps, so it shouldnt ever really hit that, but it does means that if you get a burst it could get behind.

Compact Blocks get much better compression than the old relay network in real-world testing. There are a few minor tweaks that need to happen that will take them from reasonably-often-missing-transactions to rarely-missing-transactions (including orphan pool, recent-replacements, etc).

This isnt a change from the old behavior - the old client used to send transactions it received to bitcoind locally ((re-)starting a relay network client remains one of the easiest ways to get your mempool filled after restart a node, at least until 0.14).

Compact Blocks is a far, far superior protocol to use on the wire vs the old realy network protocol, and the fact that its in bitcoind directly instead of via an external daemon means the latency of it can be much lower (it can skip, for example, the hash-message-check-message-hash stuff that happens now). Indeed, there is a drawback of occasionally having an extra round-trip from clients to relay network servers, but current testing shows that a) the improvement in average latency between the servers far outweighs most RTTs to clients (eg within a continent you shouldn't see more than 50-100ms RTT), b) its relatively easy to shave another 10-30ms (yes, really, its that bad) off the total receive->ProcessNewBlock time, which should all happen in 0.14, in compact blocks, whereas for the RN code its really not so trivial, and c) we can hopefully make the RTTs much less common with some simple fixes which the compact blocks protocol already supports, just aren't yet implemented.

Really not sure what you're trying to say here, but - New network is UDP between its own servers (dedicated 1Gbps on every link except the Beijing one, but its fast enough, too), but then Bitcoin-P2P protocol between the servers and clients. Hopefully you're using Bitcoin 0.13+ so you'll be using compact blocks, which are generally more effecient than the old relay network protocol by a decent margin (plus I have some hacks to force compact block clients into low-latency mode). If you have a server which is particularly far from the nearest relay network server I'm happy to talk about setting up something that isnt Bitcoin-P2P, and am thinking about doing FIBRE-based peering (so UDP floods for block announce).

Nope, see https://github.com/bitcoinfibre/bitcoinfibre/commit/335f87424a60af1fc02bffb7c84e90b8e2eadeca which will fast-relay compact blocks out to peers within a few ms of receiving the block if either a) you're a segwit peer or b) segwit has not activated.

Ahh, I see the misunderstanding - nope, its only Bitcoin-P2P publicly for now.

The new peering instructions are up at http://bitcoinfibre.org/public-network.html.

Yea, the HK one died after the host decided to wipe the drive on the server when the NIC failed....yes, you read that right. I didn't bother to re-install since a) I dont actually remember how to reinstall that stuff, and havent had time to look it up (FIBRE is so much easier...) and b) I want to turn the thing off for SegWit (though getting a public FIBRE network up has taken longer than expected :/.

Indeed, one of the things I'm planning on doing is listing those sponsoring on the site by putting their logo up. I assume if folks are paying 2-300$/month for something they will use it. Indeed, it was an uphill battle to get people to use it initially, but its much better these days. Bitfury still has some insanity occasionally, but I'm working with them to solve it.

Yea, they're all dedicated servers since the random lag introduced by VPSes was incredibly annoying. Sadly, for dedicated servers, unlike virtual ones, it can be hard to get them in good locations in some places (APAC especially), and its also very nice to have them all on one network, so they're all on Softlayer. They're a bit more expensive than neccessary, but they have really, really good latency between any two points on their network (with the exception of JPY<->EU as they dont have TEA bandwidth), as well as solid peering in Tokyo/HK with PCCW/Pacnet, which is important for China.

Hi folks,

I'd like to turn off the relay network on December 1st. For that to happen, I'd like to first have a public relay network of nodes based on FIBRE available ASAP.

In order to make the network much more reliable/lower-latency than the original relay network, I'd like to use real servers with good,
dedicated, bandwidth. I speced out a network which would cost about $1550/month and would have public peering nodes in Hong Kong, Tokyo, Seattle, Washington DC, Beijing, and one of London/Amsterdam/Frankfurt. Additionally, there would be (at least) one node in Siberia to utilize the TEA Cable route.

A few folks have already agreed to chip in to make this a reality, but I'd love to see a few more so that we can make it incredibly cheap for
everyone involved. If you'd like to help, I'm looking for people willing to make commitments to cover a fair share of the above costs, and would list the sponsors on bitcoinrelaynetwork.org (which would be updated with new info).

If anyone is looking to set up a FIBRE-based network, I'm happy to share the slow-as-fuck bash script which generates the slow-to-render pretty graphs on test network stats page.

If you're talking about running your own FIBRE-based network: No need to enable incoming UDP on pool servers to run FIBRE. TCP/Compact Block relay to servers only a few ms away is plenty fast, so you could just as easily put a relay network server on a separate server (and probably should, given that its still beta and based on Bitcoin Core master just before segwit was merged).

If you're talking about connecting to someone else's: I never really envisioned it as running over UDP between miners and the first-hop server. While this is definitely an option (and FIBRE has an "untrusted" mode for this), there is little advantage to spending 10ms calculating FEC data over just eating the 10ms RTT for a dropped packet or two if the servers are nearby. FIBRE is great for long-haul links, but Compact Blocks work just as well if you're talking about short-hops.

General note: FIBRE's UDP does have a shared-key HMAC in each packet, so that packets are easily and trivially authenticated.

So I'm sure everyone has seen, by now, my blog post on the future of the relay network. I do have a test-network based on FIBRE which has shown incredibly good performance, but I'm not quite sure if I will allow public connections to it (it currently only connects to random nodes on the public P2P network and to the original relay network) or allow others to use FIBRE to take up the mantle of running high-speed relay networks. For now, the original relay network continues to run and the FIBRE network running side-by-side is only making it faster!
If you want any help running your own FIBRE-based network, I'm more than happy to help out!

For those paying attention...The relay network donation address was recently nearly entirely depleted to pay for a new set of servers...The network is going to switch entirely to a new infrastructure based on bitcoind with UDP-FEC (ie no more delays from TCP lag) which allows for cut-through routing by rumoring UDP-FEC packets around the entire network before receiving the entire block. You can find the source for this patch at https://github.com/TheBlueMatt/bitcoin/tree/udp-wip...The API to it is being refactored to enable the cut-through routing optimizations, and note that it includes some LGPL components in the binary, so its probably all secretly under LGPL, but IANAL, please dont sue me.

See https://github.com/TheBlueMatt/bips/blob/master/bip-TODO.mediawiki for something similar which is more targeted at the P2P nature of Bitcoin. See-also https://github.com/TheBlueMatt/bitcoin/tree/udp-wip for something which is intended to eventually replace the Relay Network - a whitelisted-peers-only secondary relay channel that is easy for miners to use between their own nodes (ie its just an additional flag) and will also be much faster than the relay network today in its worst-case (and probably average) relay times.

Servers go down sometimes .

Until other things that are much higher priority (ie the current drama) is dealt with, I will keep restarting servers when they go down but no other changes are gonna happen...I'll revisit things in a month or three.

Yea, eu went down for a bit...should be good now.

To be fair, having a miner or  pool do it would be a significant conflict of interest. That, and I'd largely prefer to have multiple people run different networks, so I'd kinda prefer to not hand over the network as it is, and just shut it down at some point.

Development, first. There's about three months of changes that I think would be much faster that seem to work, but I'm not gonna switch the servers to using them since that would mean a ton of work debugging when they inevitably fall over in production. Eventually I'll probably shut down the servers, since it would be nice if other people ran similar networks anyway.