Quote from: JDcloudBDS on November 08, 2019, 07:26:19 AM

Blockchain Data Service - BDS

Blockchain data service (BDS) is launched by JD Cloud's Blockchain Department. It stores the linked and unstructured data of blockchain in a structured way through technical means, and synchronizes it into a data warehouse of high performace in real time.
Users can use the blockchain data query tool to accomplish simple condition query and complex SQL query, so as to help innovative enterprises who need blockchain data to run business.
Blockchian Data Service has the following advantages:

Comprehensive: provide data for a lot of blockchain programs including BTC,ETH,EOS,LTC,XRP and BCH. Also define more than 100 data indexs involving users,tansactions,blocks and so on. All data could be queired with internet available wichi is accurate and efficient.
Convenience: provide a strongly interactive BI tool for data visualization which offers real time generation of various data reports plus graphical interfaces and also support conditional query by clicking and draging.
Speciality:support using SQL language to conduct many complicated querys.
Blockchian Data Service of JD cloud is the first product in China to provide online analysis service of blockchain data.

Product Backgroud

In recent two years, with the rise of blockchain technology, the whole industry is full of contention. There are famous public chain projects such as BTC, ETH, XRP, and well-known alliance chain projects such as Fabric, Enterprise Ethereum Alliance, Corda R3, etc.
Whether public or alliance chains, in fact, each is an independent island of information, data and information can not circulate between each other. In the data-oriented Internet era today , how to integrate the data and information of these block chains together and make use of the relatively mature data analysis tools to analyze and model the block chain data has become a question that JD Cloud Blockchain Department has been thinking about.
Finally, JD Cloud Blockchian Department upholds the original intention of product design and launches the Blockchain Data Service.

Product Framework:
https://zhulang.s3.cn-north-1.jdcloud-oss.com/%E5%9B%BE%E7%89%871.png

For blockchain data, because most of the public-chain nodes are deployed overseas, the deployment of full-node synchronization in the domestic environment often results in a synchronization failure. Relying on the strong fundamental capability of JD Cloud, we deploy lots of node overseas to ensure real-time synchronization of public chain data for each independent public chain.
We observed that the top three cryptocurrencies are BTC, ETH and XRP, among which the total storage space of BTC node is the least. At present, the total storage space of BTC node is 200 GB, while of ETH node it is close to 3 TB. The largest storage space of ETH node is XRP, which is close to 10 TB, and the normal physical server is no longer available for such considerable amount of data.
In the process of analyzing unstructured data of public chain, we initially used the exposed interface of the full node of public chain to invoke the original data of the query chain through RPC. This method is convenient, but has following problems :
Data timeliness: Block data synchronized by all nodes need to be persisted to disk before it can be obtained through RPC interface.
Performance loss: In order to responds to RPC requests, full nodes need to read data from disk which will have IO loss and CPU comsuption for processing all external requests.
After careful consideration, we modify the source code of full node in public chain directly and when full nodes synchronize the block data, we directly push the data to large data analysis platform, thus ensure the timeliness of data.

Custom data
Cry