As DeFi bleeds billions of dollars annually due to token bridge attacks, a new layer-2 solution seeks a solution from within the Bitcoin blockchain.

Most bridge protocols use central storage to back assets on the receiving blockchain, creating a target spot for hackers. The lack of developer expertise compared to the complex nature of creating a bridge that works across multiple networks causes security vulnerabilities, making cross-chain bridges the largest attack surface of the DeFi space.

To make DeFi more resilient to exploits, a new approach removes this weak link between blockchains. Mintlayer is a layer-2 blockchain on the Bitcoin network that aims to connect its sidechain to the Bitcoin community with as little friction as possible. Users can build any type of existing DeFi application on Mintlayer without compromising the core fundamentals of security and decentralization that the Bitcoin community is built on.

Mintlayer aims to eliminate token bridges, the largest attack surface of DeFi, which costs users billions of dollars every year. Instead of wrapped tokens, users can exchange their BTC for tokens on the Mintlayer blockchain via atomic swaps. By not requiring any sort of cross-chain bridges, wrapped tokens or pegging mechanism to use Bitcoin on its blockchain, Mintlayer eliminates the risks associated with bridges and third-party custody.

Proving their supremacy when it comes to investor security and prioritizing investor interests is where decentralized finance has the ability to shine.

For most retail crypto investors, centralized digital marketplaces like Binance, Coinbase, Kraken, and many more have been the preferred gateway to the world of cryptocurrency trading. However, as leaked customer data like the recent Gemini case and misappropriated funds like the FTX debacle continue to plague the space, crypto investors have been looking into alternatives. According to a Delphi Digital analysis following the FTX crash, DEX tokens saw a 24% uptick, and CEX tokens saw a 2% decline.

The lack of transparency in how CEX operations are run has become a major driver for this trend shift. What’s more, a lot of platforms claim complete decentralization when in fact, there’s a centralized element to them. But do DEXs have what it takes to take over the crypto exchange space?

Are DEXs better than CEXs?

The centralized control over customer funds that companies running CEXs have combined with a lack of transparency of what goes on behind the scenes at these businesses is the most concerning aspect of centralized trading at the moment. This centralization, unfortunately, allows for the misappropriation of customer funds to happen, as was the case with FTX.

The surge of automated market makers (AMMs) was led by Uniswap, showing the crypto world the potential of DEXs. Order book DEXs such as dYdX that use zk-rollups to implement order books off-chain introduced an alternative solution to replace centralized exchanges. The uprise in DEXs is just the beginning.

While offering users more convenience when transferring or swapping tokens, centralized exchange wallets are decidedly less secure than non-custodial storage solutions. DEXs offer their users flexibility when it comes to storing their crypto tokens and do not rely on their user’s crypto holdings to generate revenue or indulge in secondary transactions.

Bitcoin is a day away from matching a nearly 10-year-old record

 I gave some of the bitcoins to my nephew as a Christmas present, and he was surprised but happy to take them.

1.The challenge for modern blockchain data stack
There are several challenges that a modern blockchain indexing startup may face, including:

Massive amounts of data. As the amount of data on the blockchain increases, the data index will need to scale up to handle the increased load and provide efficient access to the data. Consequently, it leads to higher storage costs, slow metrics calculation, and increased load on the database server.
Complex data processing pipeline. Blockchain technology is complex, and building a comprehensive and reliable data index requires a deep understanding of the underlying data structures and algorithms. The diversity of blockchain implementations inherits it. Given specific examples, NFTs in Ethereum are usually created within smart contracts following the ERC721 and ERC1155 formats. In contrast, the implementation of those on Polkadot, for instance, is usually built directly within blockchain runtime. Those should be considered NFTs and should be saved as those.
Integration capabilities. To provide maximum value to users, a blockchain indexing solution may need to integrate its data index with other systems, such as analytics platforms or APIs. This is challenging and requires significant effort placed into the architecture design.
As blockchain technology has become more widespread, the amount of data stored on the blockchain has increased. This is because more people are using the technology, and each transaction adds new data to the blockchain. Additionally, blockchain technology has evolved from simple money-transferring applications, such as those involving the use of Bitcoin, to more complex applications involving the implementation of business logic within smart contracts. These smart contracts can generate large amounts of data, contributing to the increased complexity and size of the blockchain. Over time, this has led to a larger and more complex blockchain.

In this article, we review the evolution of Footprint Analytics’ technology architecture in stages as a case study to explore how the Iceberg-Trino technology stack addresses the challenges of on-chain data.

Footprint Analytics has indexed about 22 public blockchain data, and 17 NFT marketplace, 1900 GameFi project, and over 100,000 NFT collections into a semantic abstraction data layer. It’s the most comprehensive blockchain data warehouse solution in the world.

Regardless of blockchain data, which includes over 20 billions rows of records of financial transactions, which data analysts frequently query. it’s different from ingression logs in traditional data warehouses.

We have experienced 3 major upgrades in the past several months to meet the growing business requirements:

Subscribe
Join us in showcasing the cryptocurrency revolution, one newsletter at a time. Subscribe now to get daily news and market updates right to your inbox, along with our millions of other subscribers (that’s right, millions love us!) — what are you waiting for?

2. Architecture 1.0 Bigquery
At the beginning of Footprint Analytics, we used Google Bigquery as our storage and query engine; Bigquery is a great product. It is blazingly fast, easy to use, and provides dynamic arithmetic power and a flexible UDF syntax that helps us quickly get the job done.
However, Bigquery also has several problems.

Data is not compressed, resulting in high costs, especially when storing raw data of over 22 blockchains of Footprint Analytics.
Insufficient concurrency: Bigquery only supports 100 simultaneous queries, which is unsuitable for high concurrency scenarios for Footprint Analytics when serving many analysts and users.
Lock in with Google Bigquery, which is a closed-source product。
So we decided to explore other alternative architectures.

3. Architecture 2.0 OLAP
We were very interested in some of the OLAP products which had become very popular. The most attractive advantage of OLAP is its query response time, which typically takes sub-seconds to return query results for massive amounts of data, and it can also support thousands of concurrent queries.

We picked one of the best OLAP databases, Doris, to give it a try. This engine performs well. However, at some point we soon ran into some other issues:
Data types such as Array or JSON are not yet supported (Nov, 2022). Arrays are a common type of data in some blockchains. For instance, the topic field in evm logs. Unable to compute on Array directly affects our ability to compute many business metrics.
Limited support for DBT, and for merge statements. These are common requirements for data engineers for ETL/ELT scenarios where we need to update some newly indexed data.
That being said, we couldn’t use Doris for our whole data pipeline on production, so we tried to use Doris as an OLAP database to solve part of our problem in the data production pipeline, acting as a query engine and providing fast and highly concurrent query capabilities.

Unfortunately, we could not replace Bigquery with Doris, so we had to periodically synchronize data from Bigquery to Doris using it as a query engine. This synchronization process had several issues, one of which was that the update writes got piled up quickly when the OLAP engine was busy serving queries to the front-end clients. Subsequently, the speed of the writing process got affected, and synchronization took much longer and sometimes even became impossible to finish.

We realized that the OLAP could solve several issues we are facing and could not become the turnkey solution of Footprint Analytics, especially for the data processing pipeline. Our problem is bigger and more complex, and we could say OLAP as a query engine alone was not enough for us.

4. Architecture 3.0 Iceberg + Trino
Welcome to Footprint Analytics architecture 3.0, a complete overhaul of the underlying architecture. We have redesigned the entire architecture from the ground up to separate the storage, computation and query of data into three different pieces. Taking lessons from the two earlier architectures of Footprint Analytics and learning from the experience of other successful big data projects like Uber, Netflix, and Databricks.

4.1. Introduction of the data lake
We first turned our attention to data lake, a new type of data storage for both structured and unstructured data. Data lake is perfect for on-chain data storage as the formats of on-chain data range widely from unstructured raw data to structured abstraction data Footprint Analytics is well-known for. We expected to use data lake to solve the problem of data storage, and ideally it would also support mainstream compute engines such as Spark and Flink, so that it wouldn’t be a pain to integrate with different types of processing engines as Footprint Analytics evolves.

Iceberg integrates very well with Spark, Flink, Trino and other computational engines, and we can choose the most appropriate computation for each of our metrics. For example：

For those requiring complex computational logic, Spark will be the choice.
Flink for real-time computation.
For simple ETL tasks that can be performed using SQL, we use Trino.
4.2. Query engine
With Iceberg solving the storage and computation problems, we had to think about choosing a query engine. There are not many options available. The alternatives we considered were

Trino: SQL Query Engine
Presto: SQL Query Engine
Kyuubi: Serverless Spark SQL
The most important thing we considered before going deeper was that the future query engine had to be compatible with our current architecture.

To support Bigquery as a Data Source
To support DBT, on which we rely for many metrics to be produced
To support the BI tool metabase
Based on the above, we chose Trino, which has very good support for Iceberg and the team were so responsive that we raised a bug, which was fixed the next day and released to the latest version the following week. This was the best choice for the Footprint team, who also requires high implementation responsiveness.

4.3. Performance testing
Once we had decided on our direction, we did a performance test on the Trino + Iceberg combination to see if it could meet our needs and to our surprise, the queries were incredibly fast.

Knowing that Presto + Hive has been the worst comparator for years in all the OLAP hype, the combination of Trino + Iceberg completely blew our minds.

Here are the results of our tests.

case 1: join a large dataset

An 800 GB table1 joins another 50 GB table2 and does complex business calculations

case2: use a big single table to do a distinct query

Test sql: select distinct(address) from the table group by day


The Trino+Iceberg combination is about 3 times faster than Doris in the same configuration.

In addition, there is another surprise because Iceberg can use data formats such as Parquet, ORC, etc., which will compress and store the data. Iceberg’s table storage takes only about 1/5 of the space of other data warehouses The storage size of the same table in the three databases is as follows:


Note: The above tests are examples we have encountered in actual production and are for reference only.

4.4. Upgrade effect
The performance test reports gave us enough performance that it took our team about 2 months to complete the migration, and this is a diagram of our architecture after the upgrade.


Multiple computer engines match our various needs.
Trino supports DBT, and can query Iceberg directly, so we no longer have to deal with data synchronization.
The amazing performance of Trino + Iceberg allows us to open up all Bronze data (raw data) to our users.
5. Summary
Since its launch in August 2021, Footprint Analytics team has completed three architectural upgrades in less than a year and a half, thanks to its strong desire and determination to bring the benefits of the best database technology to its crypto users and solid execution on implementing and upgrading its underlying infrastructure and architecture.

The Footprint Analytics architecture upgrade 3.0 has bought a new experience to its users, allowing users from different backgrounds to get insights in more diverse usage and applications:

Built with the Metabase BI tool, Footprint facilitates analysts to gain access to decoded on-chain data, explore with complete freedom of choice of tools (no-code or hardcord), query entire history, and cross-examine datasets, to get insights in no-time.
Integrate both on-chain and off-chain data to analysis across web2 + web3;
By building / query metrics on top of Footprint’s business abstraction, analysts or developers save time on 80% of repetitive data processing work and focus on meaningful metrics, research, and product solutions based on their business.
Seamless experience from Footprint Web to REST API calls, all based on SQL
Real-time alerts and actionable notifications on key signals to support investment decisions

The blockchain unicorn created Web3 Engine as a developer tool set targeting DeFi, GameFi and NFT products and services.

Digital asset custody platform Fireblocks has launched support for Solana, giving its thousands of users access to the developer network’s applications and infrastructure.

Beginning Tuesday, Fireblocks users will have direct access to Solana’s various decentralized finance (DeFi) and Web3 applications — a move the company says will directly benefit “alternative asset managers” and “capital market participants.”

“Thousands of businesses use Fireblocks to access novel Web3 products and services,” Solana co-founder Anatoly Yakovenko said in a statement. “Starting today, these businesses will immediately have access to the depth and breadth of the Solana ecosystem.”

The Solana integration will come via Fireblocks’ Web3 Engine, a suite of tools designed for developers building in the DeFi, GameFi and nonfungible token spaces. As Cointelegraph reported, the Fireblocks Web3 Engine was launched in May of this year. Fireblocks CEO Michael Shaulov said his company’s Web3 suite offers Solana ecosystem developers a higher level of security when developing products.

Deeply doubt that would happen
It's more than payment
and much bigger for mainstream adoption

A nice gesture to remember that day
Just keep in mind that it has been part of the history

Always glad to hear about this kind of news
Find ways to integrate itself into life

That's pretty surprisingly unexpected
It's the official accounts,right
They really need to go an extra mile

Thanks for sharing
the history has to be some indictaion

Nothing new about it
Lots of people follow suit
A minor step to the next level, maybe

It depends on you TOTALLY.

Well, so many things are emerging in our daily life. The day when everyone embrace Crypto will come early if there's more momentum going on.

Yes, we're entering the era of CBDCs.
Blockchain is the future, there's no doubt about that. Governments are catching the wave also. Plus, they don't need to print money from then on, just need IT infrastructure, which is environmental friendly.

Rug Pull is so common nowdays and people never tired of getting involved in. They trust their luck, wanna try it out, kind of like gamble, they become increasingly addicted.
I think the important lesson is to know when to stop

That sounds horrible.
19 Million US Dollar is already a large sum, not to mention Bitcoin in that quantity.
Last thing we want is inflation, it's not in the interest of the common people.

If Russia blocks access to the internet, mining difficulty might drop and it would become easier to mine Bitcoin. But price would affect demand as well, so there will be new balance. Things always have a way of working out. So don't worry.

Russian is so well known for its mining industry. As far as I know, mining is not banned in Russia, since people are still engaging in mining activities.
Anyway, put it on the legal lever may be beneficial for those big miners, expecially those close to the government.

What you described sounds like movie clips to me.
If I were you, I'd prefer to spend money during this life rather than 100 years later. Who knows what will happen next? 100 years is such a long time of period to imagine.