Gauntlet’s Market Risk Assessment

Recently, we published a formal analysis of the financial risk to participants in the Aave V1 and V2 protocols. The report expounds on methodology minutiae and specific results of our research. This post should serve as a TLDR and also give members of the community an opportunity to give feedback and suggestions about other market risk implications to the Aave protocol.

Background

At the time of writing, Aave has over $7, $8, $10 billion in TVL (thanks @Anjan-ParaFi!). With growth of this scale, mitigating financial and market risks is increasingly important as billions in customer funds are on the line.

Within decentralized finance (DeFi), borrowing protocols have four main risk factors:

  1. Security risk
  2. Governance risk
  3. Oracle risk
  4. Market risk

This post will focus on market risk, but we discuss how the Aave protocol addresses each of the other risk factors in our formal report.

Defining Market Risks

There are four primary sources of market risk within the protocol:

  1. Extreme downward price movement: Shocks to market prices of collateral that cause the contract to become insolvent due to under-collateralization
  2. Asset illiquidity and liquidator inaction: Loss of liquidity in an external market place, leading to a liquidator being disincentivized to liquidate defaulted collateral
  3. Cascading liquidations: liquidations lower external market prices which in turn lead to further liquidations (i.e. a deflationary spiral)
  4. Slashing of the safety module: Insolvency of the safety module due to extreme events where multiple collateral types concurrently fail to be liquidated

How do we test if the Aave protocol can be resilient to each of these market risks? Simulated stress tests!

Simulated Stress Tests

The main tool that we use to perform simulation-based stress tests on Aave’s Ethereum smart contracts is agent-based simulation (ABS). ABS has been used in a variety of stress test contexts, including to estimating censorship in cryptocurrency protocols, detecting fraudulent trading activity in CFTC exchanges,and stress testing frameworks from the European Central Bank and the Federal Reserve. These simulations, while powerful, can be difficult to make both useful and accurate as model complexity can make it hard to match experimental results. Careful design, tuning, and infrastructure architecture can help avoid these pitfalls and has made ABS invaluable in industries such as algorithmic trading and self-driving car deployment.

Please take a look at our report to learn more about our simulation environment!

Many elements of our simulation are designed to study the aforementioned market risks.

Extreme downward price movement
We use a multi-asset correlated Geometric Brownian motion (GBM) to simulate price trajectories. The two most important aspects of this model are asset volatility and asset correlation. In order to properly simulate market capitulation and extreme price drops, we train models on historical data with various volatility regimes. Below is the price of ETH on Black Thursday with the cumulative liquidation value on the Aave V1 protocol. Both the price path and the delay in liquidations are key inputs to our simulation.

With the abundance of cryptoassets available on the Aave protocol, it is important to quantify and understand asset correlation, which can give insight into possible contagion and correlated liquidation and insolvency risk. These are the correlation matrices for Aave V2 in March 2021 (Top) and Aave V1 in January 2021 (bottom).

Asset illiquidity and liquidator inaction

Asset illiquidity can lead to liquidator inaction because the liquidator will need to give up large amounts of profit to trade out of collateral and borrow asset positions. Below is a visualization of optimal liquidator trade sizing with respect to expected slippage in different volatilities and order sizes.

In addition, extreme network congestion will also lead to liquidator inaction. In order to simulate gas congestion and transaction delay, we construct modified and truncated normal distributions from 3 month trailing gas price data. At each time step in the simulation, we sample from the probability distribution to define the liquidator behavior.

As we increase ETH volatility in our simulations, we also increase the mean, standard deviation, and skew of the gas price distribution. Below are the daily gas percentile curves for February 2021 and generated gas curves.

Cascading liquidations
The deflationary spiral that can occur as a result of large liquidations is reproduced in our simulation by training market impact models. These models are trained on order-book (for centralized exchanges) and AMM (for decentralized exchanges) data. After a trade occurs (i.e. selling of liquidated collateral), we capture the impact that it has in the short-term and long-term.

Two factors that can affect the market impact are asset volatility and the size of the trades. We define the scaled liquidity of an asset as the average daily traded volume of the asset divided by the total collateral on the protocol. This is a snapshot of the market characteristics of each of the assets on Aave V2 in March 2021.

Slashing of the safety module
The Aave protocol has a safety module denominated in the native AAVE token which serves as a backstop for protocol insolvency. Yield seeking investors can purchase AAVE, stake (lock) it within an insurance pool, and earn a pro-rata portion of AAVE rewards (currently 550 AAVE per day).
If an Aave market reaches a state where its liabilities (outstanding liens) are greater than its assets (collateral), then the AAVE safety module covers the shortfall of all markets voted by the community.

Our assumptions for safety module slashing are very conservative: we only consider the staked Aave safety module and do not use the liquidity provided by the 80/20 Balancer pool. In addition, we make the assumption that most centralized liquidity will disappear in a slashing event. In simulation, whenever there is an insolvency on the protocol, we sell Aave into the market and model the impact with only DEX liquidity.

Metrics

We will first define metrics that will help us answer these questions in a quantitative manner.

  • Insolvent account: Any account where the total borrow value in USD is larger than the total collateral value in USD.
  • Net insolvent value: When looking at a single account, this is the net insolvent value of the account at the end of a simulation run divided by the total collateral value of the account at the beginning of the run. When looking at the entire protocol, this is the sum of the net insolvent value of all accounts divided by the total collateral value of all accounts.
  • Net insolvent value percentage: When looking at the entire protocol, this is the sum of the net insolvent value of all accounts divided by the total collateral value of all accounts.
  • Asset net insolvent value percentage: This is the sum of net insolvent value divided by the total collateral value pro-rata supply percentage for all accounts that supply a given asset as collateral. Because of the many-to-many nature of collateral and borrowed assets on the Aave protocol, attributing insolvency statistics to specific assets can be convoluted.
  • Slashing run percentage: This is the percentage of simulation runs that end with greater than 1% of the total value (in Aave) of the safety module slashed to cover insolvencies and with greater than 10% drop in total value (in USD) of the safety module.

Results

Our baseline results for almost 6000 simulation runs showed that the protocol is resilient based on our metrics (<1% net insolvent value percentage of the entire protocol and <5% net insolvent value percentage of every asset). The graphs below show the percentage of users who will have any insolvency and the net insolvent value percentage of each asset in our initial simulation runs.

The initial simulation runs also serve to identify which assets’ parameters could have the largest impact on the health of the protocol. We chose YFI to study the impacts of various borrow parameters and liquidation parameters. Below is a graph of YFI asset net insolvent value percentage as a function of correlated asset volatility and loan-to-value ratio. Note that the insolvent value percentage is monotonic in both loan-to-value and volatility scalar, as one intuitively expects.

Next is a graph of YFI asset net insolvent value percentage as a function of correlated asset volatility and liquidation threshold. Note that the choice of liquidation threshold is only important at high enough volatilities: at low volatilities, changes to the threshold do little to change net insolvent value.

This is a graph of YFI asset net insolvent value percentage as a function of correlated asset volatility and liquidation bonus. Note that this measures the likelihood of correlated cascades (e.g. the y-axis is the volatility of an asset highly correlated to YFI with different liquidity levels). This figure illustrates that correlated assets admit a `safe region’ in terms of liquidation bonus where net insolvent value stays relatively low. Unlike the previous figure, we see that there is a much stronger dependence on liquidation threshold even at lower volatility scalars.

This is the total collateralization of the Aave protocol as a function of correlated asset volatility and liquidation bonus. Note that the total collateralization ratio of the system is inversely related to the net insolvency value at high volatility levels when the liquidation bonus is above 115%.

Finally, this is the percentage of simulation runs that ended with significant slashing (>1% Safety Module sold and >10% decrease in USD value). Note that we only observe significant slashing at high liquidation bonus and volatilities and that there is a `safe region’ of moderately sized liquidation bonuses.

In section 6.3.1 of the formal report, we give a more mathematical explanation for why the liquidation bonus has this relationship with protocol safety.

Conclusions
In this report we conducted a market-risk assessment of the Aave protocol via agent-based simulations run against the Aave contracts.

We stress-tested the liquidation mechanism under a wide range of market volatility and sizing scenarios to ensure that the protocol can prevent borrowers from becoming under-collateralized in most of these cases.

We also used historical market data from centralized and decentralized cryptocurrency exchanges to ensure that assumptions about volatility and slippage are representative of real-world conditions.

We found that the protocol can withstand aggressive borrowing parameters even in extreme asset volatility and network congestion situations. However, variations in the liquidation bonuses of the protocol lead to more drastic and unexpected results, and liquidation bonus should not be the primary tool used to adjust protocol-wide market risk.

We also studied the relationship between borrower behavior as a result of borrowing parameter adjustments to understand the returns of both AAVE holders and lenders on the Aave protocol as a result of parameter changes. As market conditions change,

the optimal parameters and suggestions will need to dynamically shift as well.

Our results suggest that monitoring and adjustment of protocol parameters is crucial for reducing risk to lenders and slashing in the safety module.

The community can expect us to be more active going forward with parameter suggestions and liquidation analysis in the near future!

11 Likes

I have read through the report from Gauntlet and I must commend them on the work that they have done. Clearly it is a good start in assessing one part of the risks that Aave is exposed to. I also see that there has been tremendous and impressive work by the Risk Community in Aave. I have read what I have been able to find in this risk documentation and other areas. It is clear that this is a topic genuinely considered and taken seriously.

I do have some questions about the overall risks and it would be really helpful if someone knows the answers. I am from a background in money and risk (I am an actuary) but I am fairly new to Blockchain so I am sorry if I am asking anything silly.

  1. Is there an overall Aave Risk Register that records the risks that have been identified with their likely occurrence and impact and the approach taken to mitigate, avoid, accept or transfer? Both in terms of the protocol and our customers.

  2. The report lists four categories of risk: Security risk, Governance risk, Oracle risk, and Market risk and others are mentioned elsewhere. But are there others that are not covered by these categories? For example Regulatory risk, Reputation risk (both for Aave and for Crypto in general), Ethical risk, Operational risk, etc. I can also see other categories have been considered by the Risk Community too. Has any work been done to create a list of all the risk categories and thought given to how to manage the categories?

  3. What levers have been identified for controlling risks? Are these documented and who has control over them? What is the consensus mechanism for moving these levers?

  4. Is the Gauntlet software open source so that others can check their findings and perform peer reviews on their results? Using open source software tools encourages decentralization and promotes trust.

  5. There are types of risks borne by customers that are not borne by Aave for example Insurance Risk if they stake in the Insurance Pool. Should we consider if we have a duty of care towards customers to identify, manage, and communicate these risk and if so how do we approach their risks? For example has there been any modelling to understand the expected Insurance Pool return? How much do returns vary? Is this positive in the long-run? If so do we know what this is and should it be communicated to customers? If it is negative what needs to change? What is the probability of large losses? Do customers understand what they are investing into and can they cope with possible losses?

  6. Does the Aave Community know what its acceptable risk tolerances are both in terms of the Protocol and in terms of customer outcomes? What level of liquidations in terms of numbers and amounts, over say a one year time horizon is acceptable? Has this been communicated to customers so they know what risk they are taking on? There seems to be focus on ensuring the protocol survives – which is obviously a good outcome and completely the right place to start. But it is kind of the minimum to achieve as a foundation.

  7. Has any consideration been given to varying the Risk Thresholds by size of the customer’s deposit? Perhaps it should start high and reduce as the size of the deposit increases?

  8. Has any thought been given to vulnerable customers? For example people who are in difficult personal circumstances and or who do not know what they are doing and could lose money that they need. Or customers who may suffer from personal, mental or addiction issues? Is Aave producing good outcomes for these customers and their families?

Thank you very much for taking time to read my questions and hopefully there are answers to at least some of them. There has been a lot of impressive work by the Aave Risk Community and it is clear that you take this area seriously and want to provide good outcomes. I am more than willing to help with finding answers if needed and if there is any scope to do so.

3 Likes

Excellent report, @wfu! Thank you for putting this together for the community. It is great to see that the simulated stress tests showed protocol resilience toward insolvency. Here are some questions I had:

  1. Within the correlation analysis, did you take into account the shift higher in correlation that typically occurs with periods of heightened market volatility?

  2. In the report, you used correlation metrics and trailing volatility to estimate the price trajectories for assets that were either not liquid or not created during the historical Black Thursday market event. Given new information from the most recent drawdown we just had, did the assets you simulated in this way line up with what actually occurred?

  3. In the conclusion of the report, you suggested that ongoing monitoring/adjustment of protocol parameters is crucial for reducing risk. Have you considered what an appropriate program or methodology for monitoring and adjustment would look like? What guardrails or triggers would you suggest putting in place to ensure that we are optimizing parameters during periods of high volatility/increased correlation?

Again, thank you for the report. I look forward to your continued involvement in the Aave community.

2 Likes