BGD. Aave v3 Ethereum. New deployment vs Aave v2 Upgrade


TL;DR

After doing a deep technical analysis of the Aave v2 Ethereum upgrade to v3, we have noticed important tradeoffs that have to be made going along this upgrade path.

We would like to open a community discussion regarding the upgrade plan for a new deployment of Aave v3 on Ethereum, keeping Aave v2 Ethereum running, without upgrading.


Context

Approximately six months ago, on March 2022, Aave v3 was deployed following a multi-chain strategy: Polygon, Avalanche, Arbitrum, Optimism, Fantom, and Harmony. On some of those networks, the presence of Aave was new (Arbitrum, Optimism, Fantom, Harmony); while on others (Polygon, Avalanche), there were already existing deployments of Aave v2.

This launch process was community-coordinated:

  1. The Aave genesis team presented the development of Aave v3 to the community.
  2. The community signaled a clear approval on the forum, given the innovation and technical superiority of v3 vs v2. Licensing of the protocol was discussed and voted on.
  3. The communities of the different networks candidates for v3 presented their proposals for Aave to approve deployments there. E.g. Aave v3 on Avalanche HERE, Aave v3 on Polygon HERE, and so on.
  4. Deployment was performed on the networks decided by the community.

On the proposals of deployments on networks where Aave v2 already existed (Polygon, Avalanche), a topic that appeared was if the route should be to upgrade the smart contracts of v2 to v3 (given that it was theoretically possible to do it by modifying the v3 implementation) or to simply do a “clean” new deployment of Aave v3, and let v2 and v3 live together on those networks.

After some discussion and arguments in favor of a new deployment like THIS, that was the decision for Polygon and Avalanche. And so Aave v3 Polygon and Aave v2 Avalanche were born.

But something that was implicitly derived from different threads of discussions was that Aave v2 Ethereum, being the biggest pool of all on v2, should go through an upgrade of its smart contracts, and consequently, don’t deploy a new Aave v3 Ethereum pool, but waiting for the upgrade to be implemented. Aspects that were taken into consideration were especially the Ethereum network congestion (high cost to migrate), additional costs of managing the migration, and also that, superficially, it seemed the technical upgrade process would be pretty seamless.


The goals of an Aave v2 to v3 Ethereum upgrade

During the last months, one of the tasks to tackle by BGD on Aave has been the implementation/adaptation of the Aave v3 codebase to be able to upgrade Aave v2 Ethereum with it. We knew it was possible, even if challenging (especially in the testing phase), given the high stakes of such a “hot” upgrade of a production system, with an order of billions within.

We set ourselves certain goals to achieve with this upgrade:

  • On-chain integrators (other smart contracts) of v2 should not be affected. This means that all addresses of the protocol (Pool, Addresses Provider, aTokens, debt tokens, etc) would remain the same. In addition, all interfaces would remain completely backward compatible, with only additions for the new v3 features.
  • Off-chain integrators should not be affected. This means that, apart from the previous points of addresses and interfaces, also all events will remain compatible; again, only with new events added “on top” of the new features.
  • Introduce as less tech overhead as possible, but knowing there would be. Currently giving technical maintenance to the Aave liquidity protocol means supporting 4 different shapes of systems in production, across 10 separate deployments: Aave v1, Aave v2 Ethereum, Aave v2 Avalanche/Polygon (slightly different from Ethereum), and all instances of Aave v3.
    Even if this variety of deployments across networks and protocol versions gives a lot of value to Aave, it also introduces technical overhead (together with important DAO coordination). So our goal was to have a v2→v3 version of the codebase as close as possible to v3.

To achieve those goals, 3 main types of resources were required:

  • Engineering itself of a party knowing really deep both Aave v2 and Aave v3 (BGD).
  • Implementation of an ad-hoc test suite, specifically targeting a “hot” upgrade like this: both v2 and v3 have extensive test suites, but the implementation of this upgrade would lead to something in the middle, which should be itself tested.
  • Extensive external security procedures. Mainly adaptation of Certora rules (from v2/v3) and both code and upgrade procedures audit by SigmaPrime.

And so, BGD started working on executing and coordinating.


The challenges

As with any development, but especially with one of this magnitude, we did a deep analysis of what would be required for this v2→v3 upgrade. Initially, we identified 2 efforts that could be split into 2 sequential phases:

  • Phase I, including a “light” upgrade of the LendingPool smart contract, together with deployment of the so-called interest rate strategies smart contracts for all assets listed on Aave v2 (re-using part of them between assets).
  • Phase II, including a big upgrade of all other components, which should be done atomically in 1 governance proposal. This would include more changes on the LendingPool, aToken smart contracts, and in summary, almost all components of Aave v2 Ethereum.

Phase I is ready in practice, but during its development, and already looking to Phase II, we noticed some concerning aspects.

  • It is not trivial at all to make the v3 events compatible with v2, with some changes causing most likely problems in the off-chain infrastructure of integrators. These problems of course can be solved by supporting these integrators, but the effort on their side could be important.
  • Address and interface compatibility is achievable; strictly nothing would be “broken” for anybody using Aave from a smart contracts perspective. But the integration of the new v3 features for this v2→v3 version will be certainly different from the integration of v2, and different from the integration of v3. This means that if somebody has let’s say Aave v2 Ethereum and Aave v3 Avalanche integrated already, the integration of the v2→v3 upgraded version will be different.
  • Security procedures on this are lengthy. Apart from the BGD testing work itself, we estimate at minimum 1-1.5 months of SigmaPrime, and weeks on the Certora side; given that there is not really a precedent of a “hot” upgrade of this caliber on a DeFi protocol.
  • Some assets listed have custom implementations, like AMPL or stETH. The upgrade path requires treating them ad-hoc, which adds complexity.
  • Clearly, this v2→v3 version of the protocol will not be (implementation/maintenance wise) nor v2, neither v3. What this means is that, whenever a change/improvement is required across all v3 instances, this pool will require in some cases extra integration.

An extensive and more technical analysis of the points summarised before can be found on the following document we produced

https://wheat-guardian-dfc.notion.site/Aave-v3-Ethereum-Technical-analysis-of-Aave-v2-Upgrade-1ba8df0a7b0d4c49bfa9a86d451d88c2


Given that we care about our work being really productive to the Aave community and we really value others’ resources like security time, we started wondering how much sense it actually makes to do this upgrade versus doing a new deployment of Aave v3 on Ethereum, following the same approach as on Polygon and Avalanche, with both v2 and v3 living together in parallel.


New Deployment vs Upgrade. Key aspects

Deployment of a new Aave v3 Ethereum

  • A new version of Aave v3 on Ethereum can be quickly released. The codebase is already prepared for network-agnostic deployment, so only technicalities like the oracle denomination should be evaluated.
  • No security procedures are needed. Aave v3 is audited and formally verified, nothing else is required if a new pool is deployed on Ethereum. This means that security resources of the community (SigmaPrime, Certora) can be re-directed to other projects, and there are plenty of them coming not only from BGD.
  • The number of development resources (BGD) is significantly reduced, and will be allocated to other projects in the pipeline: generalized solution for cross-chain governance, Aave governance v3, different improvements on Aave v3, GHO, etc.
  • It gives the possibility for the community to reassess the risk configuration of the assets currently listed on v2. With an upgrade v2→v3, everything will remain the same as currently, that is not possible. It is important to highlight that this also allows for the listing of for example fewer Aave-custom implementations for some assets, like wstETH, instead of the current stETH version listed on v2.
  • Integration of Aave v3 Ethereum becomes plug-and-play for all projects who are already connected to other Aave v3 instances. Major projects around the ecosystem are fully compatible with Aave v3 (e.g. Dune), so this approach will make their integration straightforward.
  • GHO’s first facilitator can be a “fresh” instance of Aave v3, which makes things simpler for testing and simulation purposes pre-deployment.
  • For the future, a single Aave v3 codebase. This means less maintenance overhead, and the possibility to do batch updates on all networks, with the same code. This means reducing future technical debt in the future, which translates to fewer operational expenses by the DAO.
  • Aave v2 will not be dismissed, or deprecated anyhow. Aave v2 is a really solid protocol, working well and extensively battle-tested. Therefore, there is no need to anyhow declare it as dismissed, only not doing incremental additions, like new asset listings (that will go to Aave v3 Ethereum, with even better risk assurances).
  • A pure Aave v3 deployment will be cheaper gas-wise. Aave v3 introduces important gas optimizations, which become quite important in an environment like the Ethereum network. An upgrade of the contracts would still keep those optimizations, but there would be some sacrifices on them, mainly because of compatibility with the previous storage layout (e.g. “live” user positions on Aave v2).

Upgrade of v2 smart contracts

  • “Automatic” migration of positions. Users of Aave v2 would automatically become users of v3, as the addresses of the v2 smart contracts will be exactly the same as v3. This means no transaction or migration procedure is required from the Aave v2 Ethereum users.
  • Some integrations will just work out of the box from the existing v2. Especially those integrations on the smart contracts layer will just work without doing major changes to their on-chain infrastructure.
  • Some off-chain integration will also work without changes. Most probably not so many in this case, but those that only touch certain parts of the system could work potentially without change.
  • No liquidity segregation. If doing an upgrade, all the Aave available liquidity at the moment will remain exactly the same. While with a new deployment, technically the Aave v2 liquidity will be separated from the liquidity of the new Aave v3. It is important to highlight that everything will still be Aave liquidity, and the access to it by integrators/users will work almost exactly the same way.

How would it look the procedure to deploy Aave v3 Ethereum?

  1. The community discusses on the forum the path to go.
  2. A Snapshot is voted on for Aave to make a decision. Assuming positive result on deployment a new Aave v3 Ethereum for example’s sake. The Aave community authorizes BGD to deploy.
  3. Discussion on the community for all initial configurations of the Aave v3 Ethereum. Including the initial set of listings and all other protocol parameters.
  4. In parallel with 3), some improvements are to be applied to all v3 instances, including an audit pre-deployment.
  5. Smart contracts deployed: everything not active, paused.
  6. Governance proposal for the community to activate the pool.

BGD conclusion and next steps

The duty of BGD with the Aave community is, apart from delivery on development, to give a critical/expert point of view of topics related to the tech of the Aave ecosystem.

Our opinion is that the advantages of doing a new deployment of an Aave v3 Ethereum holistically and importantly outweigh those of upgrading the current Aave v2 smart contracts.

That being said, our ethos is to be always transparent with the community, and submit these types of important topics to the discussion here on the forum; for the community to actually do a decision, using our analysis as a base.

We ask everybody to participate in the discussion of this important topic, supporting one direction or the other. After some days, we will also create a Snapshot vote for the community to decide on how to proceed.



Links

14 Likes

Thanks the update!

Reading through the situation, I fully agree with BGD’s conclusion and new V3 Ethereum market makes sense. I have 2 questions:

  1. Would V3 launch with all the assets that are currently listed on the V2 market? Would that fall into the scope of the initial V3 deployment or would all assets need to be re-assed and go through governance?
  2. Would there be a plan to provide a tool to make it easy for users to migrate their positions?

Again, thanks for updating the community. Great to have this level of transparency.

Looking forward to V3 on mainnet!

2 Likes

Having contributed to the BGD analysis and originally lead the v3 development and deployment, I agree with the broader conclusions of the OP - I do believe that a new deployment would be more beneficial for the Aave ecosystem than a migration in the long term. A few important points remain open in my opinion - first of all, we will need to attract liquidity in the new market - the community will likely need to come around with a short term, well targeted liquidity mining program for v3.

Regarding the point of the assets listing @G-Blockchain, I think it would be a great opportunity to clean up the market from illiquid/deprecated assets (eg fei, UST, knc) and better retarget some assets using isolation mode (something that couldn’t be done if the co tracts were to be updated)

4 Likes

Agree with BGD that new deployment seems more suitable. Lower long run operating costs and gas advantages seem sufficient of a reason to support new deployment rather than upgrading. It also adheres to opt-in upgradability, which I think is a positive factor from decentralization standpoint and will also reduce execution risk.

The main negative is that some idle capital may stay stranded in the v2 protocol for a long time. For example, Aave v1 still has over $20 million in stablecoin deposits held within various OG iearn finance curve pools. The saave and aave Curve pools (using v2 stablecoins) have around $33 million in deposits, and it seems possible that a portion of this may not end up migrating over - in addition to other capital that remains in v2 due to inertia / lost keys / etc. V2 is still a great standalone protocol so it’s okay that some money will stay there, but liquidity fragmentation is not great for Aave user experience.

Some ideas for how to help minimize liquidity fragmentation and incentivize migration to v3:

  • Short term (eg 3 month?) liquidity mining for key assets on v3, such as stablecoins and ETH
  • Only support new asset listings on v3, not v2
  • After 3-6 months from v3 launch, increase reserve factors on Aave v2 (raises protocol income while incentivizing migration)
  • After 3-6 months from v3 launch, moderately reduce liquidation thresholds across Aave v2 markets (reduces management overhead, allowing Gauntlet and others to focus efforts on v3 while limiting insolvency risk)
  • After 3-6 months from v3 launch, consider removing security module protection of Aave v2 Ethereum market
6 Likes

Upgrading the Aave Protocol Ethereum V2 to V3 market directly would have indeed created some seamless experiences for the users, at the same time creates an event that would require lot of security procedures and places substantial assets at risk. While I would personally like to see the upgrade to happen, I lean towards finding more benefits on creating a completely new market for the following reasons:

  • Saves time and resources (can be allocated in other parts of the protocol)
  • Gives the Aave community also an opportunity to review the assets within the protocol
  • Assets could support GHO growth strategy (inc. liquidity mining)
  • Most importantly, new market would be more safer solution and adds less risk
5 Likes

Regarding your point of “well targeted liquidity mining”, it’s been discussed in a few posts and the community are somewhat against it, including myself.

I believe there could be other solutions first before looking at liquidity mining. There could be a gas refund program for the first 3 months of V3, where the treasury could refund users gas spent in migrating their positions fro V2-V3. This would mean the cost to the user migrating would be 0 and in the long run less costly for the treasury, especially at today’s gas costs.

Hello @G-Blockchain . Regarding your questions:

  1. It is up to the community and I think it will require a separate discussion if the new deployment path is chosen. But we believe that at least it is necessary to 1) as @Emilio points out, re-evaluate the listing of some assets that don’t add much currently to Aave v2 Ethereum or are directly deprecated and 2) certainly re-configure the assets to do good use of the new v3 features (ceilings, isolation mode, eMode, etc). We highly encourage a collaborator on the risk side like Gauntlet to kickstart this discussion, if/when the Snapshot vote to choose the new deployment direction passes.
  1. Yes, that is on our radar too, given that engineering resources will be freed from the upgrade. Something to take into account is that for migrations v2 → v3 to happen via flash loans, highly probable will be necessary to have some liquidity already bootstrapped into v3. But we will check a bit better this during the following days.
2 Likes

I share the sentiment on liquidity mining in general, but in this case if we want to attract liquidity to allow people to migrate some kind of incentives will be needed. Refunding gas fees wont help if there are no assets to borrow in the first place, and on V2 there are some VERY large positions to migrate. Capital will be needed in any case.
A well targeted, short term liquidity mining program narrowed to a few key assets is nothing like a full fledged long term liquidity mining.

1 Like

Gauntlet is supportive of a new deployment for Aave V3 Ethereum. As noted in the comments above, there is significant complexity shifting over a large system like Aave.

Should the community decide to choose the path of a new deployment, Gauntlet looks forward to supporting the community around initial listings of the V3 Ethereum market and configurations as mentioned above.

I’m very pleased by this direction – Volt Protocol originally planned Aave as its first yield venue integration but deferred because of the planned full-market upgrade of v2 to v3. Aave v2 and Compound v2 remain the most battle-tested lending markets and highly liquid yield venues in DeFi. Launching new code as a standalone product allows user choice and potentially more total users between the two pools than would exist after a forced migration.

Gradually ungoverning Aave v2 would be great to see from the perspective of a potential integrating protocol.

3 Likes

A Snapshot vote has been created, with voting starting tomorrow 4th October, and ending 7 days after, on 11th October.
https://snapshot.org/#/aave.eth/proposal/0x584eb4e0f79e1d9dcdd99b3a0c831bfc3c654af3f8f619d5f68eae23cd9cb149

We highly encourage the broad Aave community to participate on the decision.

7 Likes

To follow up on the above, Gauntlet has provided recommendations here to kickstart community discussion.

1 Like

As an update regarding Aave v3 Ethereum, we have published an update of the current status on another thread [ARC] Risk Parameter Recommendations for Aave V2 ETH (2022-11-22) - #27 by bgdlabs. Adding it here too for visibility.

The pending items regarding Aave v3 Ethereum at the moment are:

  • The security providers of the community (Certora and SigmaPrime) are already reviewing the 3.0.1 changes that need to be applied to the v3 codebase pre-deployment, with a timeline of 1-2 weeks.
  • The Aave IPFS user interface should be verified to understand if it will properly support Aave v3 Ethereum (we will coordinate with @AaveLabs, the maintainer, on it). This should be doable in the following 2 weeks.
  • Once/if the proposal about initial listings HERE is approved, we will do an extra review on all of them (e.g. oracles). We expect this to be possible in the following 2 weeks to.
  • The Aave governance proposal payload for the activation/deployment of the system needs to be built, among other things, connecting all the components together. This depends on all previous points, but standalone is not time-consuming for us.
5 Likes

We would like to update the community on the status of the Aave v3 Ethereum release.

  • The updates on the codebase of Aave v3.0.1 have been finished.
  • The update/payloads of Aave v3.0.1 periphery are ready. We will be proceeding with the activation via the required proposals on the existing v3 this week (targeting tomorrow).
  • Given its presence on the initial set of assets to be listed, we have produced an oracle adapter contract for wstETH.
  • Given the number of assets to be listed at the same time on v3 Ethereum, we have been working on a generalized listing engine for Aave v3.0.1 instances, that will simplify the procedure not only in this case but in following listings of any contributor. We will be publishing it this week.
  • Security procedures can be considered finished too, with the different reports to be published during the week. @AaveLabs has ordered an extra audit by Peckshield, which has been successfully finished too.
  • We have coordinated with @AaveLabs for the testing/QA on the Aave IPFS interface, being successful at the moment.
  • The process of release has been delayed, because, even with the technical aspects nailed down, Gauntlet stopped the process of initial risk parameters recommendations, requiring the other risk provider of the community (@ChaosLabs ) to take over the task. We have coordinated with Chaos, and they will have an update for the community soon.
  • We have been working on a migration tool Aave v2 → v3, that if approved by @AaveLabs , will be introduced in the Aave IPFS interfaces, both for Aave v2->v3 Ethereum, and Avalanche/Polygon.
    Some last points need to be addressed, but ideally, the tool could be launched even prior to Aave v3 Ethereum, for users of Polygon/Avalanche to use. We will release a more extense explainer on the tool soon.

In summary, all technical aspects including security reports are to be finished during the week. The lack of initial risk parameters recommendations has delayed slightly the timeline, but on the positive side has given some extra room to have everything around more solid.

Given that is not realistic to have a Snapshot approval on initial risk parameters before the end of the week, we advocate rolling out Aave v3 Ethereum beginning of January 2023, given that doesn’t seem to be reasonable to introduce important decisions for the community the last week of the year 2022.

10 Likes

Great update @bgdlabs. It is very exciting to see v3 coming to Ethereum.

Great to see Chaos Labs stepping up to help bring v3 to Ethereum.

For those that don’t know, there is this discussion thread on the initial eModes parameters which is linked below:

The community can participate in the discussion whilst we await input from @ChaosLabs on the topic.

2 Likes

An update for the community about the Aave v3 Ethereum release process.

  • The Snapshot vote to decide the initial assets to be listed is finished, with the community choosing the reduced initial set proposed by Chaos Labs. Aave v3 Ethereum will start with WBTC, WETH, wstETH, USDC, DAI, LINK, and AAVE.

  • Only 1 eMode will be active: ETH-correlated assets, including WETH and wstETH.

  • All assets apart from wstETH and WBTC will have a “pure” Chainlink price feed connected. For wstETH and WBTC we will use price adapter smart contracts (the latter following the decision on https://snapshot.org/#/aave.eth/proposal/0x2b95df8c7b2141acdead83beb4cdede9624bb2a4c74f873a5727e2a54f322525.

  • The initial interest rate models will follow those present on v2 Ethereum assets and other existing instances of v3, especially because v3 models are slightly different from v2 (mainly in what concerns stable rate). In the case of ETH and wstETH (enabled to borrow, even with a pretty low cap), the rate model will be the one defined on https://app.aave.com/governance/proposal/131/.
    We highly recommend that risk/treasury entities of the community put a more granular definition of rate models as high-priority items. Still, we don’t consider this a blocker for activation.

  • The v2->v3 migration tool is in its last steps, being actively reviewed. We expect it to be out with v3 Ethereum activation, but it is possible to have some delay on it.

If the final review process goes smoothly, we expect to be creating a governance proposal for the activation of Aave v3 Ethereum this week.

16 Likes

An update for the community about Aave v3 Ethereum.

The technical setup for the Aave v3 Ethereum activation is finished, and, in order to have the voting running during the week, we plan to submit the on-chain proposal on Sunday: voting will start Monday and if the proposal passes, execution will be Friday 27th.

The final AIP description can be found HERE, with all the information about proposal payload.

We will keep updating the community during the following days on all the details of Aave v3 Ethereum, but we are open to any question here in the forum.

16 Likes

Following the correction done on the WETH rate strategy here, we have updated both the AIP description (also fixing a typo) and the proposal payload to be submitted.
No further changes have been applied from the previous version.

2 Likes

The governance proposal to activate Aave v3 Ethereum has been created.

Voting starts in approximately 24h https://app.aave.com/governance/proposal/?proposalId=147

6 Likes

The governance proposal has passed successfully and has been queued.
Aave v3 Ethereum will be live after the execution in less than 24h.

8 Likes