Price Oracle — A Must Have Infrastructure

14 min readFeb 20, 2021

Overview

With the exponential growth of decentralized finance protocols over the past 12 months and the growing interest from other corners of the cryptocurrency market to DeFi, we have witnessed a tremendous shift in crypto market narrative from centralized to decentralized finance.

However, for DeFi to truly achieve its goal to disrupt traditional finance and transform the entire financial industry to be more efficient and transparent, the space still has a long way to go. And one of the key obstacles that it needs to overcome to inch closer to this goal is the oracle problem.

Price Oracle: A must-have infrastructure

The problem with the blockchain world is that Dapps cannot directly communicate with off-chain information to enable further innovations such as dispute settlement, liquidation, risk management which all require off-chain data to be fed into the smart contract. This is where price oracles serve as an essential component of the DeFi space — price oracles are the “connectors” between the on-chain and off-chain worlds, delivering data generated by real-life, off-chain events on-chain and thus trigger a series of smart contract execution.

Decentralized financial services offered by the current mainstream DeFi platforms such as MakerDAO, Compound, dYdX, etc. rely heavily on the price oracles to provide the data necessary to carry out smart contract operations such as determining liquidation prices, margin trading, and lending. Price oracles act as a reliable source of information to feed these critical data to these DeFi platforms. As such, price oracles are an integral part of the decentralized financial services infrastructure.

Problems With Current Oracles

Decentralized finance is unconventional, and we have observed a few differences when comparing it with the traditional finance industry.

Lack of viable options to conduct proper risk management

In traditional finance, when institutions talk about investing, trading or having market exposure, the first thing they think of is usually risks. Quantifiable risks form the foundation of the financial market and are crucial for market participants to make informed investment decisions. In DeFi, however, we have barely seen any analysis and recognition of such risks, not to mention incorporating quantifiable risks into the DeFi infrastructure. In other words, as of today, market participants have not been able to comprehensively understand risks in DeFi, nor can they properly calculate and hedge them.

Time in the on-chain world is discrete rather than continuous, measured on a block by block basis

In the off-chain world, time elapses continuously and the market has evolved over time into having a complicated price discovery mechanism based on the continuous timeline. As such, a market’s price discovery is almost always more efficient in the centralized world than the decentralized one.

However, in the on-chain world, time elapses discretely and the smallest unit of time in the blockchain world is known as the block time.

Due to this time structural difference, we argue that DeFi applications should not compete with their centralized counterparts for being the destination of price discovery. Instead, these applications should bring their unique competitive advantages to the game, creating exclusive experiences in the decentralized world. Unfortunately, the current DeFi ecosystem is still building infrastructure based on the centralized mindset.

Existing designs of automated market makers (AMMs) rely on arbitrageurs to ensure that prices on AMM DEXs do not deviate from actual market prices

Because the traditional financial market has become fairly efficient through years of evolution, arbitrage opportunities are hard to come by. However, the current AMM-based DEXs heavily rely on arbitrageurs to help approximate true market prices, implying an inefficiency in the design of the DeFi infrastructure.

Liquidity providers (LPs) are not compensated enough for having market exposure on volatile assets

There are various market frictions in the centralized trading world such as latency and transaction fees. These frictions are also true in the decentralized world, and DeFi LPs are not being compensated enough for bearing the cost incurred by these frictions.

Given the concerns listed above, we believe that the blockchain infrastructure, specifically price oracles, should be built upon the following design principles.

a) Enable computable risks

b) Consider time as discrete rather than continuous, with the smallest unit of time as blocks

c) Meet the non-arbitrage condition

d) Incorporate time delay, market volatility and transaction costs when compensating the LPs

Overview of the Price Oracle Sector

The demand for oracles heightened in recent months as more DeFi protocols are being launched.

As seen in the charts below, between May 1, 2020 and September 1, 2020, the 30-days rolling gas fee for oracle project Chainlink showed a 32.2x surge, while the standard gas fee on the Ethereum blockchain only jumped by 26.9x during the same time period.

Chainlink and another oracle project, NEST Protocol, were among the top 10 Ethereum gas gazers on September 1, 2020, ranked #5 and #7 respectively. Both protocols rose to prominence within a short period of time — on May 1, 2020, Chainlink was ranked #11 while NEST Protocol was not on the list.

The gas distribution during the period between August 15, 2020 and September 15, 2020 also shows that NEST Protocol and Chainlink are the only two non-DEX projects in the top 10 Ethereum gas usage leaderboard (excluding Tether USD and SmartWay Forsage, which is a pyramid scheme).

Unlike in yield farming, there is limited hype in the oracle sector, especially with NEST Protocol. Even with the recent DeFi hype, the google search index of keyword “blockchain oracle” was near zero for 80% of the days over the past three months. Being the must have infrastructure in the industry together with a low google search index, implying huge potential for further development.

Oracle protocol review

There are essentially two types of oracles, centralized and decentralized ones.

Coinbase Price Oracle is a notable example of a centralized oracle. It takes Coinbase Pro’s API as the data source, then excludes data points with significant deviation. DeFi projects wishing to request data feeds can make calls to Coinbase Price Oracle with public key verification enabled.

Although centralized oracle is an easy way to fulfil the off/on-chain data gap, we have only seen a few DeFi projects utilising it, largely due to security concerns — an attacker can control the oracle data sources and then broadcast malicious information to a smart contract that relies on the data, thus contaminating the decentralized trading environment.

On the other hand, decentralized oracles have been widely accepted by the DeFi ecosystem. DeFi applications such as lending, derivatives, and some DEXs all need a reliable oracle to access price data in order for them to function properly and fairly.

In this report, we investigate four decentralized oracle projects: Chainlink, NEST Protocol, Band Protocol, and Tellor, each of which have been created based on various design principles and with different levels of decentralization.

· Chainlink provides price feeds for 55 trading pairs, including 30 fiat pairs. More than $2.3B locked values in crypto assets are referencing Chainlink price oracle as of September 14, 2020.

· Band Protocol aims to provide a price feed service on multiple public chains, including Ethereum, Polkadot, and Cosmos. Beyond cryptocurrencies, Band Protocol can also connect to trusted data sources (i.e., weather, lottery, and unemployment rate) through traditional Web APIs with customised oracle script.

· Tellor takes the old fashion Proof of Work (PoW) method combined with the staking concept to maintain data quality. Users can request data by adding TRB token as a tip.

In a macro view, the three oracles mentioned above have structured their price generation model as shown in the process below.

Due to the on-chain validating requirement, these decentralized oracles may fail to update when the market experiences extreme volatility, like the one on Black Thursday, due to network congestion and extreme market conditions.

The process of on-chain validation needs voting by transaction, which consumes extremely high gas fees during peak blockchain network activities. Even if sufficiently high gas fee has been utilised, voting can still encounter blockchain propagation issues.

NEST Protocol, on the other hand, proposes a different way to solve the problem of providing prices on-chain through a decentralized incentive solution called quotation mining. In a nutshell, the quotation mining system consists of two types of participants

1: Makers which consist of miners and verifiers

2: Callers. Anyone can become a miner to provide quotation and pay commissions in return to receive NEST tokens. On the other hand, verifiers can challenge the miner’s quotation by staking 2x equivalent amounts of asset to update the price if the previous quotation price deviates from the true market price.

Each of the current DeFi projects brings some elements from traditional finance to smart contracts, with the lending sector considered to be the largest by total value locked as shown on the left. However, we argue that the majority price oracles are too centralized which only report data on short tail assets. Out of the top 100 cryptocurrencies on Coingecko which have a combined value of $328B, roughly 22% of the $328B assets has not yet been reported by decentralised oracles.

We believe the DeFi ecosystem should not pick and choose on what asset data should be reported by the price oracles. Instead, these decisions should be made by the community. However, when reporting data on long tail assets, we run into the problem of attacks. The maximum penalty to maliciously attack an oracle node by reporting incorrect price data is the amount of assets that the attacker staked. The monetary value could be significantly less than the potential profit earned if a successful attack can be executed, generating an incentive for malicious attackers to exploit the protocol.

NEST protocol presents a properly designed structure to eliminate these potential attack vendors. A malicious attacker may tamper with a normal quotation price, however, the attacker must stake 2x the amount of assets in order to challenge the previous quotation price. The attacker either leaves significant arbitrage opportunities to the market or must continue to self report and continue staking 2x of the previous amount of assets in order to sustain the attack. The costs of an attack can become exponentially large within an hour of the attack and raise to almost the total number of ETH in the entire market. This design makes NEST protocol attack resistance in the sense that it can completely eliminate dishonest players in the system.

Price feed comparison

Oracles are in charge of price feeds. The donut chart below demonstrates a simple comparison about methods for oracles to generate data and how those data are validated. Chainlink uses price data from nodes with LINK tokens as collateral. Though anyone can run a Chainlink node, the Reference Data Feed is generated still by permission. Currently, Chainlink has 21 trusted parties for its ETH-USD pair with 0.5% deviation threshold.

In Tellor’s ecosystem, users can request price feeds by giving TRB tokens as incentives. Miners quote price with PoW competition; and the median of the first five price feeds will be saved on-chain. Users are able to dispute the price feeds within one day after price feed generation.

NEST Protocol allows miners to participate quotation mining by staking base token and quote token with an off-chain price; i.e., staking 10 ETH with $3,800 USDT means Ethereum (ETH) has a current market price of $380. The arbitrager (Taker) can challenge the quoted price by performing a swap for both ETH and USDT within 25 blocks (i.e., T0 ~ 5 mins) if the arbitrager believes that the price quoted by the miner is incorrect.

In the meantime, the arbitrager needs to provide an updated quote and stake 2x amount of asset. When no one challenges the quotation price within 25 blocks, effectively meaning that the price meets the non-arbitrage condition, this quotation price then becomes the NEST- price, which can then be used by other DeFi applications. On the other hand, there is no verification process for other price oracles and thus no way to ensure the price data reported by other oracles meet the non-arbitrage condition.

In addition, NEST protocol can achieve on-chain price updates for every block under the most efficient scenario. Since the smallest unit of time is the block time in the on-chain world, price updates on every block is the most efficient way that can be achieved.

The unique risk factors value proposition

Price delay and price deviation are two key risk factors presented in all oracle infrastructure. Network congestion or a lack of price feed can cause price delay causing price deviations. These two risk factors can influence the performance of other DeFi platforms relying on these oracles.

Unfortunately, there is no way to eliminate price delay and price deviations due to the fact that the off-chain world runs on continuous time, but the on-chain world runs discretely by block.

As a result, after an asset’s price undergoes the process of being transmitted through the oracle from off chain to on-chain, it also becomes different from the quoted price of the off-chain exchange.

In fact, the transfer of asset experience from off-chain to on-chain will produce three different prices with small deviations: exchange quotations, oracle theoretical feed prices, and on-chain quotations (true price generated by Oracles). Behind the difference between these three prices are two easily overlooked risk factors: 1: delay 2: volatility.

Delay and volatility

For all price oracle protocols, market volatility matters. Since the on-chain and off-chain world runs in parallel with different time structures, price differences are naturally presented between the off-chain and the on-chain world due to time delay. However, when the market volatility is higher, price differences can experience some even larger gaps.

Data analysis

This section presents a data-driven analysis of the differences in ETH/USDT prices between price oracles (i.e., NEST Protocol and Chainlink Oracle1) and Huobi Global exchange, using historical data ranging from August 30, 2020 to September 16, 2020.

1: Chainlink launched ETH/USD aggregation while NEST Protocol provides ETH/USDT reference price. In this analysis, we assume the difference between USDT and USD is negligible.

We discovered that the ETH/USDT price from the oracles matches Huobi Global exchange’s off-chain price data closely with some extremely small deviations.

Then we further calculate the spread between Chainlink’s and NEST’s ETH/USDT prices and exchange price. Most of the time the NEST Protocol’s price spread (red dots) is smaller than Chainlink’s (blue dots). The absolute number of Chainlink’s average price spread ($0.494) is 31x as much as NEST’s ($-0.016).

This finding affirms that the NEST Protocol’s price data can be more properly referred to in the decentralised world based on its accuracy.

The results of the spread analysis indicate that the price spreads between September 2 and 6 were relatively large, which we believe was related to the higher gas fee and ETH/USDT price volatility on the Ethereum network during the time. For Sep 2 2020, the Ethereum average daily gas price is roughly around 480 Gwei.

We have further analyzed the standard deviation of 1440 minutes rolling price spread and the results have shown that the standard deviation of the Nest’s spread was only 0.21% to 1.30%, while for Chainlink it ranges from 0.97% to 2.01%, again we have verified that the NEST Protocol has a more reliable quotation design.

We extracted data between Sep 5 and Sep 14, 2020 for deeper analysis. Both NEST Protocol’s and Chainlink’s highest absolute number of average daily spread prices occurred on Sep 5, 2020. ETH/USDT quotation price from Huobi Global exchange fell by more than $76 on that same day. While Sep 14, 2020 presents a relatively low price volatility market.

The below analysis shows that Chainlink’s reference prices (red dots) provide much less price data than NEST Protocol, which means Chainlink has a lower frequency of quotation under a stable market, due to its design.

Chainlink is set to trigger the oracle updates when the off-chain price fluctuated for over 0.5%. Otherwise, there will be no price update for as long as 10800 seconds. While in NEST Protocol, anyone can become a miner providing quotations to earn NEST tokens. With a higher frequency of quotation, Nest’s reference price tracks more closely to the Huobi Global’s than Chainlink’s.

The analysis of price spreads also confirms the above view. The deviations of the two oracles’ price spreads from zero is different under different market conditions. On Sep 05, 2020, when the market was in a higher volatility, the deviation is larger than that under the lower price volatility market.

Under the same market condition, the deviations from zero for Chainlink’s price spreads (blue dots) are worse than NEST Protocol’s (red dots) for most of the time.

Our interim analysis shows that prices provided by NEST Protocol match more closely with those from the centralised exchange. In contrast, Chainlink as one of the price oracle infrastructures demonstrated subordinate performance in terms of price reliability.

What does this mean for the future of DeFi?

The current DeFi ecosystem can be described as CeFi running on blockchain, although the space continues to strive for decentralization, innovation, and permission-less. We believe that the current DeFi market structure can be considered as DeFi 1.0, where some of the most critical legos pieces in the DeFi blueprint such as lending, trading, derivatives, price oracles, etc have been built to support the early stage ecosystem in the past few years.

We have demonstrated that current DeFi price oracles can be more accurate and reliable to support further innovation in the space. Major players such as Chainlink offer a solution to solve the most important issue in the decentralised world — connecting off-chain data with on-chain data. However, its reliability, accuracy and the decentralised nature are called into question when compared with the NEST oracle supported by in-depth data analysis.

As DeFi grows, however, DeFi 1.0 no longer satisfies all the market demands, especially those coming from sophisticated market participants. As such, we need DeFi 2.0, or what we call, CoFi, Computable Finance.

To resolve the problem that current financial risks in DeFi lack computability, a redesign of the price oracle infrastructure is likely part of the solution. At the end of the day, a sustainable DeFi ecosystem will be the one where market participants can be compensated by bearing certain risks, and these risks can be computable in such a way that they allow market participants to properly quantify hedge these risks if necessary.

In the CoFi world, we envision the reshape of the most important DeFi infrastructure — the price oracle — entirely from the ground up to attract meaningful market attraction, deliver accurate and timely data information, enable computable risks, and meet the non-arbitrage condition.