Aave v2/v3 security incident 04/11/2023

yeah sure. that is how the protocol works. if there is a bug then freeze associated assets. i have no problem with this. with respect what you said is very obvious. i don’t think anyone using an aave forum doesn’t know this. we are into crypto bro.

i would just like to know if i will lose my locked in stable interest rate when the function is disabled. and if so are there any other options? of course i don’t want to put funds at risk etc.

it is great how fast the team move to freeze assets like this.

i want to hear it from the horses mouth. i.e. @bgdlabs will those that locked in stable interest rate loses this rate?

We appreciate the great initiative taken to address this vulnerability in a timely manner and report to the DAO swiftly.

We appreciate the prompt response from @bgdlabs and the Guardians in addressing the situation.

Chaos Labs is actively monitoring the markets and the protocol and will provide recommendations given changing market conditions, as necessary.
We are working alongside other contributors to provide risk-related recommendations regarding the necessary updates.

As an update on the incident:

  • Since our first communication on Saturday, we have been focusing our effort on re-evaluating everything surrounding the vulnerability reported. Protection of the protocol and reducing users’ impact are the highest priorities for us, as service providers to the Aave DAO.

  • In addition to the measures taken on Saturday, after further research and for caution, we have advised the Aave Guardian to escalate protections on the previously only frozen assets: now the following assets are paused too:

    • Aave v3 Polygon: USDT, USDC, DAI, EURS.
    • Aave v3 Optimism: USDT, USDC, DAI.
    • Aave v3 Arbitrum: USDT, USDC, DAI, EURS.
  • In parallel, we have designed a new mechanism to be introduced on all networks where protections have been activated: the Liquidations Grace Sentinel.
    Whenever an asset is paused, no action can be done on it by design of the Aave protocol, including liquidations. Once the asset gets unpaused, all actions will be available again, which means that positions that couldn’t be refilled or repaid before, would be under immediate liquidation conditions. The Liquidations Grace Sentinel is a type of emergency oracle (Aave Guardian controlled) that allows to activate a “liquidations grace period” for any asset listed in a pool.
    Whenever this grace period is on, even with the asset unpaused, liquidations involving it will not be allowed.
    In the current Aave v2 Ethereum paused scenario, the idea will be that, if the market becomes really volatile affecting Aave positions, the risk providers of the community (@Gauntlet, @ChaosLabs) will be able to define a grace period of for example 1-3h after the unpause, on which people will be able to refill/repay their positions and avoid liquidation.
    To be clear, pausing is part of the design of a system like Aave with its risks/advantages, but we believe this gives immediate value and doesn’t compromise decentralization, as it is in practice a more granular way of pausing/unpausing.

  • Additionally, as permanent remediation and protection, we are submitting a governance proposal to upgrade the Stable Debt Tokens across all Aave instances where stable rate is active.
    This will have the effect that no more stable debt can be minted in the future, which we think is the safest direction and aligned with the community strategy (no stable debt has been enabled in v3 instances for some time).
    To answer some concerns raised in this thread: current stable rate positions will not be affected. However, we also think that in the future, the DAO should consider migration of user positions from stable to variable, as if not, there is important inconsistency and asymmetry.
    IMPORTANT. The moment of this governance proposal execution will mark when assets can be unpaused.

  • For the previous 2 measures, we have involved both an extra reviewer (@AaveLabs ) and a security service provider of the DAO (Certora), to have maximum security assurances, in addition to rigorous internal testing from BGD.

  • For a matter of good DeFi community ethos, we have checked all Aave forks (v2/v3) and advised protection measures for all those exposed to the vulnerability. Even if we don’t have any responsibility to them (we are not providing services), we think the Aave community should show good values, as leaders in the space.

We will keep informing the community as frequently as possible.

14 Likes

How long is it expected to take to resume paused stablecoin withdrawals on aave v3 Polygon?

Hi @Neil, it will be until the proposal we described regarding Stable Debt Tokens is executed.
So 6 days from the moment the governance proposal gets created, if governance approves it.

thanks. can you please explain this some more? i don’t understand what you mean by inconsistency and asymmetry?

1 Like

Why are repayments disabled? For people with substantial loans with variable rates, you force them to pay insane rates without any chance of closing the positions. And now you are saying this will take at least 6 days.

2 Likes

Insane rates are only for some days, i’m pretty sure they are doing the best to do… they probably have no choice.

@snowfox2012 whenever a pool is paused, no action is enabled, for a matter of protection. Given the decentralized Aave governance, the 6 days are a limitation that simply can’t be skipped, as they are on-chain configurations.

The rationale for the pausing is simply that, after evaluating potential attack vectors, it is the only mechanism that gives 100% assurance.

Following the previous communication, we have created a governance proposal to upgrade the stable debt tokens, which will allow for the unpause of all the affected pools.

Voting will start in ~24h

https://app.aave.com/governance/proposal/?proposalId=359

1 Like

The current borrow interest rate on EURS on polygon is 122.9%. I fully understand to protect from the vulnerability that positions had to be frozen. I don’t understand why interest rates were not also frozen, right now people are stuck with these absolutely exorbitant borrow rates. They should have at least been capped right?

2 Likes

there is no built in mechanism to freeze interest rates. It’s 6 days and everything will be restored, understand that this was an emergency situation and everything that could be humanly possible to restore normal operation asap has been done.

4 Likes

Are the pools still perfectly safe or has something changed? I mean first frozen now paused…

Aave v2/v3 Security Incident Risk Analysis

Summary

On 2023-11-04, v2 Ethereum was paused and certain assets on V3 Polygon, Arbitrum, Optimism were frozen in order to protect the protocol from a vulnerability. As a result, positions may become at risk for accruing bad debt, given that liquidations / debt repayment / collateral refill have been paused.

The situation presents a number of uncertainties with regards to the amount of time v2 will be paused and spikes in volatility and breakdown in asset correlations. Gauntlet computes potential insolvencies under the following scenarios. We include 2 week to unpause scenarios to be prudent in light of other potential delays, and for greater color on how potential insolvency and liquidations may evolve during a pause.

  • Time to unpause v2 is 1 week, volatility is 1x historical 1-year volatility.
  • Time to unpause v2 is 2 weeks, volatility is 1x historical 1-year volatility.
  • Time to unpause v2 is 1 week, volatility is 2x historical 1-year volatility
  • Time to unpause v2 is 2 weeks, volatility is 2x historical 1-year volatility

For each scenario, we run 1000 simulations, in each simulation we generate 15-minute returns, extend over 1 or 2 weeks, and compute distribution of insolvencies. The simulations assume -

  • (no stablecoin depeg) stablecoins (USDC/USDT/DAI/etc) do not change in price
  • (no LST depeg) STETH and WETH price returns mirror each other

We first discuss simulation results, and then touch on our view on the grace period that BGD mentioned previously.

Aave v2 Simulation Results

At a high level, most potential insolvency that may arise could come from borrows against stablecoin collateral, and most potential liquidations may arise out of decorrelation between WETH and WBTC.

Insolvencies_95 and liquidatable_amt_95 represent insolvencies and liquidatable borrows at the 95th percentile out of 1000 simulations. We find that at the 95th percentile, supposing volatility were what was observed in the past year, potentially $5k may become bad debt. Note that Aave has sizable reserves and treasury funds

days_to_unpause volatility_multiplier insolvencies_95 liquidatable_amt_95
7 1 5k 32m
7 2 180k 35m
14 1 240k 35m
14 2 700k 37m

Analysis of higher insolvencies under more pause delay and higher volatility reveals most insolvencies come from stablecoin collateralized CRV borrows. Some simulations model CRV prices increasing, so towards the tail end those CRV borrows may become insolvent.

Should we exclude the CRV borrows -

days_to_unpause volatility_multiplier insolvencies_95 liquidatable_amt_95
7 1 5k 32m
7 2 12K 35m
14 1 12k 35m
14 2 50k 37m

The largest contributors to liquidations at risk come from WBTC borrows. This account becomes eligible for liquidation with a 4% downwards move in WETH/WBTC ratio, and has frequently topped up positions in the past month.

Aave v3 Simulation Results

On v3 Arbitrum, Polygon, Optimism, and Avalanche, USDT, USDC, DAI, EURS have been paused. Both nonstable-collateralized stable debt, as well as stable-collateralized nonstable debt may be at risk, should market move significantly in either direction.

Assuming stablecoins will become unpaused in 1 week, simulating a 2-std move down and up reveals larger potential liquidation volume with a downwards move. At this 95th percentile, insolvency risk remains small.

chain direction new_insolvencies new_borrows_liquidatable
arbitrum down 0 860k
arbitrum up 0 360k
avalanche down 0 1.6m
avalanche up 600 240k
optimism down 0 41k
optimism up 0 14k
polygon down 0 600k
polygon up 900 560k

Conditions for grace period

Determining the viability of the grace period to allow users to repay debt / top up collateral involves tradeoffs between user experience and potential insolvency that may occur during the grace period. Our simulations reveal that under current loanbook, conditions, and governance procedure, adding a 3-hour grace period for users to readjust positions does not add excess risk.

However, for thoroughness, we outline a couple characteristics that can affect the viability of the grace period. We then define two conditions in which we would recommend disabling the grace period.

Characteristics

  • (Buffer to insolvency) Let B be the set of borrows that will become insolvent with a 5-std 3-hour move. If sum(B) > K, where K is some insolvency tolerance, then reconsider grace period.

    • If an excess number of liquidatable positions have an increased chance of evolving into bad debt, liquidating them, rather than having a grace period, may improve risk.
    • As an example, our simulations show that on the weekly time frame, an additional pause of 1 week increases 99th percentile insolvency by ~$250k (excluding large CRV borrows, see above simulation results).
  • (Increased volatility) Rolling market volatility on 15 minute period leading up to grace period is above n*observed_volatility, where n is a volatility multiplier, then reconsider grace period.

    • Increased volatility leading up to grace period may be sticky and induce higher volatility during the grace period, which may affect position buffer to insolvency.

Based on the above characteristics, should either of the following two conditions be met, we recommend disabling the grace period, if the community finds the parameterization acceptable.

Condition A

  • (Aave v2) Buffer to insolvency, K = $250k, 5-std 3 hour move
  • (Aave v3) Buffer to insolvency, K = $25k, 5-std 3 hour move

Condition B

  • (Aave v2) Buffer to insolvency, K = $250k, 2-std 3 hour move
  • (Aave v3) Buffer to insolvency, K = $25k, 2-std 3 hour move
  • (both) Increased volatility, rolling market volatility > 3 * observed volatility.
    • As an example, WETH observed volatility over the past year has been ~45%. Should WETH rolling market volatility over the past half-day be > 135%, coupled with lower buffer to insolvency, this may signal need to deactivate the grace period.
    • This would have been breached ~ 7 times in the past year.

We pick $250k for the threshold K for Aave v2 as it represents the additional insolvency risk at the 99th percentile incurred over an additional delay of 1 week, ex ante. Risk exceeding this level at activation of grace period may indicate deteriorated market conditions, in which benefits of liquidation may outweigh impacts to users. We proportionately scale down the threshold for Aave v3 deployments based on TVL.

Price, Rolling standard deviation over half-day, WETH

Next Steps regarding Market Risk

  • As mentioned, should either of the above two Conditions be met, we recommend disabling the grace period, if the community finds the parameterization acceptable.
  • Welcome community feedback.
6 Likes

They are fine, the additional pause is out of abundance of caution

1 Like

I believe you mean the bridged versions of those assets, and not the native ones – e.g. USDC native on Arbitrum is not and will not be paused, correct? If there is potential to freeze the native assets too, will there be any advance notice?

Since the native USDC does not have a stable borrow APY, it will not be paused.

2 Likes

Hey, I just checked my AAVE account and there is big funds missing from my deposited funds (about 0.028 BTC and 0.01 ETH).

There was no liquidation as my borrowed funds (USDC, DAI) are still the same amount. My health factor is 0.99 now and I can’t repay debts or anything.

What the hell is happening and please can somebody help me? This looks like a nightmare.

I am using AAVE on Polygon and borrowed stablecoins (USDC, DAI) for my deposited funds (WBTC and WETH).

I can see the transactions in Polygonscan (“Token Transfers ERC20”).

For example this is the TX Hash for my gone 0.028 WBTC:

Please tell me that this will be fixed and I am not losing a big amount of my crypto savings because of a bug bounty program?

1 Like

@PolyMika this can’t be related to the actions taken by Guardian. It seems you have been a victim of approval by mistake to a malicious entity.
You should remove your approvals with a tool like https://revoke.cash/ as soon as possible

1 Like