[ARFC] BGD. Aave v3.4

[bgdlabs]

Summary

Introducing Aave v3.4, a new version of Aave v3 (currently v3.3) including the following changes and features:

• Migration of the custom GHO on v3 Core to a standard a/v Token, with no migration required by users.
• Addition of Multi-call support on the Aave pool.
• Introduction of a new Position Manager role, to a limited subset of actions, optionally configurable by every user.
• Multiple code-size, gas, math precision, and miscellaneous optimisations.
• Removal of the direct flash loan fee injection to suppliers to reduce liquidity index increase complexity, and overall, the security model of the protocol. Opening for the DAO to re-distribute the fees in an async manner with suppliers and/or Umbrella stakers protecting the asset.
• Upgrading the core protocol smart contracts to Solidity 0.8.27.

Context

The development cycle of upgrades on a protocol like Aave is, by itself, full of nuances and strategic decisions. Sometimes, the upgrades are oriented to optimisation of the smart contracts infrastructure, and sometimes more oriented to enabling new features and improving in user/developer experience of the protocol.

Historically, the major “sub-versions” of Aave v3 focused on the following:

• Aave v3.1. Infrastructural, very oriented to security improvements, with the introduction of virtual accounting and the deprecation of stable rate borrowing.
• Aave v3.2. Feature-rich, focusing on the introduction of Liquid eModes, which radically changed the usage and configuration of assets listed on Aave pools.
• Aave v3.3. Again, infrastructural, but this time focused on optimising liquidations and starting to account for bad debt accrued on the protocol (deficit).

v3.4 is a bit of an exception, as it has a pretty good balance between UX/DevEx-oriented changes and infrastructure. The rationale is that given the criticality of the liquidations component touched on in v3.3, we decided not to include certain features and optimisations we had in a very advanced stage of development and pack them together in v3.4.

Aave v3.4 features

1. Standardisation of a/v GHO tokens on v3 Core

Since the listing of GHO on Aave v3 Ethereum as a directly mintable asset via a facilitator, we didn’t feel too comfortable with the approach, as it added certain complexity to the protocol.
This feeling got confirmed on the different upgrades of v3, as GHO became the permanent exception case that usually needed to be handled ad-hoc.

Here is a quick summary on how GHO on Aave v3 Core Ethereum looks at the moment:

• GHO aToken (aGHO). It is a special implementation that has very little in common with other aTokens, more oriented to trigger various hooks on actions, instead of being an accounting/transferability layer. These inconsistencies materialise in a different utilisation model in the pool, the inability to flashloan GHO natively on Aave, and multiple other minor quirks.
• GHO vToken (vGHO). While superficially working similarly to standard vTokens, vGHO also has some special mechanics that lead to complexity and potential bug surface.
  In contrast to other vTokens, the debt does not accrue as yield on the aToken (as aToken totalSupply is always 0), but instead accrues only on the vToken and is collected by the Aave Collector on repayment.
  This behaviour is required in order to support the discount model, but adds again very important complexity, which, for example we noticed in practice when improving liquidations on the v3.3 release. Additionally, this adds overhead on the treasury management and peg control, as it creates the concepts of realised and non-realised GHO fees.
• GHO on v3 Core is the only exception for an asset not using the virtual accounting introduced on v3.1, with no plans for this to change anytime in the future.

GHO post-v3.4

On this v3.4 proposal, GHO on v3 Core changes the following way:

• The minting model will follow that of the Aave v3 Prime Ethereum with the GhoDirectMinter: a smart contract that can supply liquidity of GHO to the pool by minting it, but technically outside of Aave v3.
• Different from currently on v3 Prime, and to not change the behaviour of GHO on Core, only the GhoDirectMinter will be able to supply liquidity of GHO on Aave.
  This will be achieved by having a 0 supply cap on GHO, which is skipped by the GhoDirectMinter exclusively. It opens to aGHO transfer scenarios (e.g., if the Collector could decide to transfer aGHO), but that should not create any issue.
• The interest rate remains as it is, totally static (not based on utilisation), and changeable via governance and stewards.
• aGHO yield accrual will work exactly as with any other aToken, continuously, and not only on repayment.
• The native discount model is removed, as it can be achieved in the same way by having GHO rewards to specific vGHO/stkAAVE holders on top, with only very small differences on debt accrual dynamics, which in practice make no difference historically on GHO borrowing positions on Core.
  The most natural option is following the anti-GHO approach approved on the new Aavenomics, for stkAAVE holders to get anti-GHO, which can be used to repay the GHO debt, factually being a discount. Important to clarify that this will not be in sync with v3.4, with anti-gho potentially being released before or after.
• No users’ migration of any type is required. On the v3.4 upgrade, the GHO model of v3 Core will change under the hood, but all user positions will not be affected.

The benefits of these changes on the technical/security side are very important:

• All reserves will have virtual accounting enabled, with no exception.
• Related to v3.3 deficit management, burning bad debt no longer relies on special logic for GHO that discounts the protocol fee in some special cases.
• GHO flashloans are enabled, eliminating the need for custom adapters, and in most cases making the flash-mint facilitator obsolete (but can be kept if the community wants).
• Even if only affecting the GhoDirectMinter and the Collector in the average case, now for vGHO balanceOf(user) (similar to aGHO) will act as any other reserve, high-level being scaledBalanceOf(user) / normalizedDebt.
• Umbrella will not need special handling of the GHO case.
• Further changes on v3 will become way simpler.

---

2. Multicall support on the Aave Pool

Multi-call support is the ability of a contract to allow native usage of multiple of its functions in one single transaction, one after the other. For example, being able to do supply + borrow, without requiring another contract on top doing under the hood the Pool.supply and then the Pool.borrow.

This has been a frequently requested feature on the Aave protocol; however, in previous iterations, we discarded it because we perceived extra complexity that we were not comfortable with.
With the type of non-invasive approach to multi-call we introduce on v3.4, we now think it really can bring all the value without compromising the security model of Aave v3 anyhow. Additionally, this will exponentially improve the experience post-v3.2 Liquid Emodes, opening to a myriad of new flows and use cases.

---

3. Aave Position Manager role

When analysing the needs of different integrators after the release of Aave v3.2, it became clear that it could be important to allow a user to authorise a third-party to the following actions on its behalf:

• Enable/disable an asset as collateral in the user position.
• Enter an eMode (or switching from one to another).

Aave v3.4 introduces the concept of Position Manager, which has the following characteristics:

• Any user can optionally configure one or multiple Position Manager addresses by calling the approvePositionManager() on the Aave v3 Pool. Same for removing authorisation.
• These addresses will have the capability to do the previously commented actions on behalf of the user, with all protocol validations still applying as usual.

The Position Manager becomes especially interesting when combined with multi-call in the pool, opening ever more use-cases and flows.

---

4. Misc improvements and optimizations

4.1. Removal of the “unbacked” concept

Unbacked supply was part of a feature called Portals on the original Aave v3.0. Eventually, even if potentially useful, the feature was never used, but it occupies an important amount of contract size while also costing gas on most of the pool transactions.
Therefore, in this version of the protocol, we decided to drop all the related logic to make room for other improvements. The feature might be added back in a future iteration if a use-case emerges and/or code-size becomes less of a concern(e.g. via Fusaka/EOF).

4.2. Multiple configurations changed from variable to immutable

On the current v3.3 iteration of the protocol, there are multiple configurations defined as variables, but that historically have a “constant” nature.
To simplify the gas and reasoning models around them, v3.4 changes the following configurations from variable to immutable.

• Interest rate strategy address. While all reserves already share the same interest rate strategy smart contract, in Aave v3.3, the strategy is still defined per reserve, which leads to one unnecessary storage read per touched reserve on user actions.
  Given that there is no realistic need any time soon to change the interest rate strategy for only one asset but not for others, in Aave v3.4, one immutable variable RESERVE_INTEREST_RATE_STRATEGY was added to the Pool contract.
• Incentives controller address on a/v Tokens. While all reserves already share the same Incentives Controlled address (the contract handling supply/borrow incentives), in Aave v3.3, the Incentives Controller was defined in state per token, which leads to unnecessary storage reads per touched reserve(s).
  This had some historical reasoning of being able to customise rewards accounting contracts per asset, but it is a behaviour totally deprecated with no plans to appear any anytime soon.
  Therefore, in Aave v3.4, one immutable variable REWARDS_CONTROLLER was introduced on the IncentivizedERC20 base class.
• Pool address on ProtocolDataProvider. Up until now, the ProtocolDataProvider was doing a call AddressesProvider.getPool() to fetch any data, but given that the addresses provider and pool proxy pair will never change, it is way more optimal to simply make the pool reference on the data provider immutable, and this way adding extra gas savings across the board.
• Treasury address on aToken. Currently, the treasury on each aToken is stored in storage, although it is the same for each aToken, even across pools in the same networks. This causes unnecessary storage reads on liquidations and mintToTreasury, with the first having high gas sensitivity. Therefore, in Aave 3.4, the treasury was moved to an immutable.

4.3. Migration to Errors

Aave has historically used string error codes as opposed to Error signatures, because there was a preference for require(cond, error), which did not support signatures in previous compiler versions.
As on Aave v3.4 the codebase was upgraded to solc 8.27, require(cond, Error()) is now supported. This greatly improves UX, as explorers and simulations now show helpful errors opposed to cryptic numeric Error codes. At the same time, the change results in minor gas & codesize savings across the board.

While this change could be breaking for anyone relying on exact error codes, it is important to note that:

• Every single Aave release since v3.0 has had breaking changes in regards to error emission (either due to new Errors or the order of Errors)
• We checked all the major integrations and did not find examples of people relying on exact error codes.

4.4. Refinement of rounding/precision on user actions

Similar to the majority of protocols dealing with any type of mathematical operations on-chain, Aave v3 has certain inherent imprecision, generally affecting the order of magnitude of +/- 1 wei (last decimal).
Considering the extensive list of security procedures applied to Aave v3 over the years, we are confident this imprecision doesn’t pose any important problem.

However, on v3.4 we are improving the rounding logic on user operations to follow the design approach of the protocol (and industry in general) of being more explicit rounding on the side of the protocol, as on the 4626 standard.

4.5. Flash loan fees model simplification

On the Aave Pool and, more specifically, the flash loan feature, there is a fee which currently has 2 components:

• The flash loan fee to the protocol _flashLoanPremiumToProtocol. The percentage of the total flash loan fee that goes as fees to the protocol Collector itself.
• The flash loan fees to aToken holders (_flashLoanPremiumTotal - _flashLoanPremiumToProtocol). The percentage of the flash loan fee that gets automatically accrued into the liquidity index, so factually “spread” as yield amongst all suppliers.

From a technical and security perspective, the model is sound and functional. However, the component of the flash loan fee going to suppliers creates a highly important consequence on the Aave system when looking from a fundamental point of view.

In Aave, yield/debt growth happens in practise by the so-called “indexes” (liquidity index, variable borrow index) increasing. Being a yield-oriented architecture, what is expected is that as debt gets accrued on the borrow side, the debt index increases, and this reflects on the liquidity index also increasing, to “forward” the yield from borrowers to aToken holders.

High-level, this feels like it should be the only gateway for the liquidity index to increase, however, there is one single exception on Aave v3: the flash loan fee going to suppliers.

Aside from being the only exception, the mechanism of “injecting” the flash loan fee into the liquidity index of the aTokens is very different morphologically to yield accrual:

• It is “fast”. It happens immediately during a flash loan, and there can be multiple “injections” of flash loan fees during the same block. In comparison, the increase of the index based on yield is “slow”, as it depends on the time passed and the interest rate.
  For example, if an asset has a yearly 5% supply rate, the rate per Ethereum block is approximately 0.001%. However, one single flash loan of the whole liquidity available would be 0.05%, 50 times more, and that could potentially be repeated multiple times in the same block.
• It is quasi-unbounded. Meaning that the higher the flashed amount, the higher the flash loan fee, and is only bound to available liquidity on the pool, which technically can be increased by supplying just before the flash loan.

These characteristics of the flash loan fee, and even considering countless other protections like virtual accounting, make the security profile of the flash loan feature more dangerous by default than yield accrual limited by time.

Because of the previous, on v3.4 we remove completely the component of flash loan fee going directly to suppliers, with the total flash loan premium becoming equal to the flash loan premium to the protocol.

The consequences of this are:

• Yield accrual becomes the only “entry point” for liquidity index increase. This is extremely valuable from both a security and a technical perspective.
• High-level assumptions don’t change significantly. The DAO can still spread to suppliers flash loan fees if required, just via other methods like on-chain rewards, Merit, or anything else.
• The DAO can choose on top to spread rewards as incentives to the users that will carry the risk of the aTokens: Umbrella stakers.
• Alternatively, the DAO can simply choose to completely zero the fee, but this would just be a separate configuration, not having any consequence on the codebase.

4.6. Misc

• Solc version used on the Aave protocol upgraded to 0.8.27.
• Gas usage of executeUseReserveAsCollateral was greatly reduced by optimizing storage access.
• Gas usage improved by shuffling virtualUnderlyingBalance into the position of pre-3.4 unbacked.
• Minor code style improvements on executeRepay.
• Usage of imported events from the interface contracts instead of re-declarations.
• Skip unnecessary calculations on a subset of transfers.
• Self-liquidation is now forbidden (BREAKING). While this is a breaking change, it’s unlikely to affect anyone, as there are essentially no on-chain traces of people relying on this functionality.
  The rationale is that even if a user has monitoring capabilities/tooling to avoid liquidation, the optimal path is repaying with collateral (or just repaying), not self-liquidating.
• SafeCast was upgraded from Open Zeppelin v4 to v5. The main difference is the usage of Error signatures, reducing the code size of various contracts.
• VersionedInitializable now bricks the initializer on the implementation, so implementations no longer have to be initialized in order to prevent malicious initialization.
• ScaledBalanceTokenBase changed the .balance storage from 128 to 120 bits. This was done to align storage across all tokens (currently on aAave on Ethereum mainnet uses 120 bits of storage), as 120 bits is more than enough.
• Improved the accuracy and gas consumption of the calculateCompoundedInterest function without changing the formula. Credit to @Certora on this simplification.

FAQ

I’m borrowing GHO on Ethereum, do I need to do anything with v3.4?
No, the migration is totally non-invasive.

So I will not have a GHO discount on the protocol after v3.4?

Not natively, but in practice, you will with the new anti-GHO proposed on the aavenomics: once that mechanism is activated, by holding staked AAVE token, you will receive anti-GHO that can be used to “burn” GHO debt on Aave v3 Ethereum core.

The discount “accrued” up until the v3.4 upgrade proposal will be applied to your position.

Security procedures

Security procedures of Aave v3.4 will start in the following days, and as usual, we will update the forum post once they are finished pre-release, together with the final codebase.

Next steps

1. Create an ARFC Snapshot for the upgrade of all Aave v3 instances to Aave v3.4, after some days of discussion in this forum.
2. Advance on security procedures for v3.4.
3. On-chain AIP creation, for the Aave governance to upgrade all pool instances across all networks.

[heyjared]

Supportive, well detailed.. thanks for house cleaning upgrades with this

[stani]

Thank you for putting forward the proposal. Previous upgrades were needed, i.e. 3.1. related security upgrades, 3.2 liquid eModes and 3.3 to support the Umbrella integration into the Aave Protocol. However, given that existing protocol is stable and previous security upgrades have been done, and given how much value at stake each upgrade of the protocol touches (30-35B) and the fact that V4 is around the corner (future evolution of Aave), I don’t recommend to proceed with this current upgrade given they don’t materially change main usage of the protocol, thus the risk reward is not there. If there is need to upgrade some of these features that are useful and more trivial such as multi-call, could be implemented. However, I do think these development resources should be rather used on other, more high value / low risk initiatives instead, given how much value is currently at stake and concerns each upgrade creates to the users and stakeholders.

[bgdlabs]

Thanks for the feedback @stani. Given that the upgrade strategy is a pretty important topic for the DAO, we believe it is a good idea to elaborate on it, and also some extra high-level strategic topics on the mentioned v4.

We don’t agree with the risk/reward equilibrium of v3.4 and following upgrades not being worth it for a matter of risk/reward. Aave v3 is indeed critical software, but the understanding of upgrades as somehow precisely because of that is unreasonable for us.

Some arguments:

• DeFi protocols are in an early stage, and precisely systems like Aave (double side, akin to lending) require improvements over time.
• We have designed each upgrade with extreme care in all senses, with quite frequently the biggest technical decisions being about what to NOT include, or what to remove to simplify flows of the protocol.
• Every upgrade has been reviewed by multiple security-industry leaders, some selected by the DAO directly, like @Certora, an extra ones by us, taking into account their experience and profiling. Software security is not a 100% certain field, but undoubtedly, upgrades at this stage are substantially safer than 3, 2, or a year ago, because all procedures continuously improve, and human contributors become more and more aware of all nuances with experience
• We carefully monitor the needs/requests of users of the protocol, both end-users and other operational contributors; features are not decided without carefully studying those.
  More precisely, v3.4 is directly inspired by important needs/feedback of the community, including security collaborators, for example:
  • Changes to the model of GHO. Overall, everybody in the community is dissatisfied with the custom nature of GHO in Aave v3 from all angles: for treasury contributors, it is difficult to handle the realised versus unrealised nature of the GHO debt.
    Additionally, and even more important, all entities that have been involved in the security of the protocol believe “custom GHO” adds massive complexity to the system, for little benefit: it overcomplicates reasoning about development and invariants of the system, which affects everything else. Testimony of it is extensively elaborated on @StErMi 's review on v3.3.
  • Multi-call. It is a “simple” feature, but highly requested for people to have “natural” flows of interaction with Aave, for example, supply collateral and borrow assets (any number of both) in one transaction.
  • Position Manager. Similar to multi-call, it opens new cases highly requested by the community and integrators, for example, handling on behalf of the user eMode switching, to change collateral exposure between, e.g. ETH LSTs/LRTs.
  • Flash loan fee simplification or rounding improvement. Both we and other teams like @Certora have spent countless hours reasoning about both topics, security-wise. This simplification is a total game-changer for the security of the system, as factually will mean there is no protocol entry point for “injection” leading to inflation-like attacks.
  • Removing self-liquidation. Only opens to problems, multiple attacks (most recent these last days with similar dynamic on MIM) are partially based on this concept.
    Additionally, adds overall complexity to Aave liquidations.
  • Removal of the “unbacked” concept. Never used, it only adds complexity on development and opens up unknown attack vectors security-wise.
  • All others are similar.

---

Regarding Aave v4, the community has not taken any decision to replace Aave v3 with v4, only to approve the development of v4. We think that decision should be more explicit, with the community understanding the very important implications of it, and voting on Snapshot.

Some aspects to consider on the v3 and v4 side:

• In terms of being conservative security-wise, no matter if v4 simplifies different parts of v3, it is a system built from scratch, and there are very strong arguments to argue that, because of that, it is way more uncertain than the existing v3, living in production with high stakes for years.
  DeFi users trust Aave and v3, which is out of any doubt at this stage. The community and various stakeholders should evaluate what it means from a strategic perspective to deprioritise the maintenance and improvement of the system they use and trust.
• Aave v3 is a fully functional system and we are fully confident in its lifetime potential.
  We do not doubt that v4 can potentially have its place in the ecosystem, but that doesn’t mean v3 is some type of bad system that should be replaced.
• For the last 3 years, a lot of components and procedures in this DAO have been built around Aave v3, arguably the majority of them: risk vertical, type of security contributions, governance frameworks, automation, tooling, treasury.
  We believe any type of v3 “deprecation” will have massive effects on everything built, negative in its majority, and definitely causing millions of dollars in expenses.
• Aave Labs has developed/is developing fully on its v4. We value our independence and fully respect that of others; because of that, we didn’t want to influence announced v4 features. However, we were not aware that the plan or intention is to factually replace v3 with v4, somehow soft/hard migrating users, and we simply don’t agree with it
• Almost any type of project on the Aave DAO requires a fully comprehensive plan on how the project fits into the ecosystem (product lines, service providers), and nothing has been disclosed by Aave Labs during the initial stages of v4 development. We think this is already causing problems as seen here, and it will create a situation like it happened with v3.0 and with GHO: a product will be released, and as the strategy is not clear at all, everybody contributing to the DAO will be in a situation of uncertainty.
• Our mandate as service providers to the Aave DAO is to make Aave systems as good as possible technically, obviously respecting other contributors, but not getting blocked by separate developments.
  This means that no matter other Aave pool implementations, there is no reason for features not to be included on the live system.

In summary, if required, we are happy to elaborate even more on each single aspect included in v3.4, but we believe they are very reasonable, worth it, and supported by other contributors.

On the v3/v4 discussion, the community should first understand, second decide on the consequences of going in the direction of total v3 deprecation in favor of a v4.

[EzR3aL]

But aren’t especially the GHO changes very good and important making it easier for the DAO?
And on another note, for now we don’t know when v4 will be released. Could be 1 week, could be 6 months.

And v3 in its current form is the saftest version we have. V4 is new and completely changed, so it’s rather more risky than v3.

So in my opinion anything that helps the DAO to integrate and push GHO is helpful and we shouldn’t waste time not using these upgrades to it.
The migration from v3 to v4 will likely also take months before we can say it’s bigger than v3.

[Emilio]

I share the feeling that this upgrade is somewhat less relevant compared to the previous ones which were great, and to proper evaluate if the risk reward is worth it, i will save my decision for when the code of 3.4 is publicly available, as i want to evaluate the impact on the codebase. While some of the proposed features i can imagine are relatively small and easy to implement in isolation like the multicall and the position manager, some others like rounding changes, the GHO migration and compiler version migration sound relatively challenging from a security standpoint. While i understand the position of @bgdlabs and i too see the value of upgradeability in a complex system like V3, we should also be very clear in recognizing how even one simple mistake is enough to kiss goodbye to the whole thing. This i think all contributors on Aave understand very deeply, and that’s one of the reasons why Aave has been fault proof, for the most part. Still, we had some security issues related to the stable rate mechanism that dated back to the first iteration of Aave, and nobody was able to find even with the code being live for over 5 years and after hundreds of audits and god knows how many thousands of black hats looking at the code. This is not meant to scare anyone, just to remind everyone the reality of the technology we work on - in a system with 24/7 liveliness, public code and no chance of addressing issues promptly due to governance, one mistake is all it takes. While i have full confidence in the ability of @bgdlabs of addressing changes securely and in our security SPs, i also keep strongly in mind past experiences and events affecting both us and other projects and i’d still avoid updates if not absolutely necessary. We are humans, and mistakes will happen, no matter how good we are. In any case, i am looking forward to the code to be able to take a more informed decision.

–

Note in regard to V4, as i am the main engineer behind it and i feel like i need to intervene. While V3 is great, V4 was designed and developed to address almost all of the aspects in which V3 could be stronger, from a capital efficiency, risk and security perspective. These aspects we all know them, we have been through challenges in the past years were V4 features would have helped immensely. I think this is undeniable and pretty much recognized by everyone who has been introduced to V4, including in the original governance post, recently @Certora and @StErMi (very soon also @bgdlabs and the rest of the SPs will get access) and also pretty much globally acknowledged on X, where i had a series of threads explaining V4 more in detail. Additionally, V4 offers a host of new features and the architecture allows for novel usecases and a variety of new options that on V3 are simply not doable (for example the multi spoke architecture).

Almost any type of project on the Aave DAO requires a fully comprehensive plan on how the project fits into the ecosystem (product lines, service providers), and nothing has been disclosed by Aave Labs during the initial stages of v4 development. We think this is already causing problems as seen here, and it will create a situation like it happened with v3.0 and with GHO: a product will be released, and as the strategy is not clear at all, everybody contributing to the DAO will be in a situation of uncertainty.

Historically the process flow of releasing a new protocol iteration has been to deploy the new protocol instance, wait enough time for the system to be deemed secure, and then activate the migration procedures. Given all the benefits that V4 brings over V3, it would be in my opinion a strategic mistake and a waste of resources not to follow the same approach. This is of course a decision the DAO needs to take, i have no doubt V4 will “cannibalize” V3 anyway in the same way V3 “cannibalized” V2 given that it’s simply much better. Hopefully once @bgdlabs will get access to the code and have a full picture they will get a better view of the potential and we can start working together on new technology and novel features.

Regarding this:

For the last 3 years, a lot of components and procedures in this DAO have been built around Aave v3, arguably the majority of them: risk vertical, type of security contributions, governance frameworks, automation, tooling, treasury.
We believe any type of v3 “deprecation” will have massive effects on everything built, negative in its majority, and definitely causing millions of dollars in expenses.

Everything will be done with the goal of keeping as much as the current infrastructure compatible with V4, of course. In case components won’t be needed anymore they can just be deprecated in regard to the new protocol iteration, same way as it normally happens in any technology stack when new versions are released.

[sid_areta]

This is our understanding of and view on the situation:

• Constantly upgrading smart contracts poses security issues to the ecosystem - the point is well made by @stani and @Emilio that even with as robust a security process as Aave and BGD Labs currently have, some issues could slip under the radar, and upgrades should happen only when absolutely necessary (like the Umbrella activation); they shouldn’t just be routine. Where we’re lacking a bit of clarity here is the nature of these upgrades and how important they are. Only then would we be able to understand the real need for them at this point. As such, there needs to be a clearer explanation of the “why” for these upgrades and the benefits users will get that are absolutely critical.

• It seems as if a lot of the upgrades mentioned here may well be incorporated in Aave v4 - please correct us if we’re wrong. After reading Emilio’s comment, it seems as if we’re pretty close to v4. That is going to be a big launch / upgrade in itself, and as such, it may not be worth it to upgrade these sensitive contracts prior to v4. However, we do see merit in including some changes that are relatively less risky and can improve user experience. It may be worthwhile for BGD and Emilio to align on these in the context of v4’s upcoming feature set and identify which changes would be most appropriate to make at the moment.

Of course, it’s great to see BGD constantly improving the Aave protocol, and we can’t thank them enough for that. The work they do is probably the best in the ecosystem on protocol development and maintenance, and we’re only asking these questions because upgrading contracts is (even in the best of conditions) a risky endeavour.

[bgdlabs]

Thanks for the feedback @sid_areta. To clarify specially on your first point:

Protocols like Aave are not “static” in nature: the market they interact with changes over time, new assets are listed, the machine hosting them changes (the EVM has radically changed since v3.0 deployment), the system gets activated on completely different networks, unfortunately, new security attack vectors appear, etc.
We have a very firm stand that, knowing that, deploying protocols in an immutable manner is plain and simple irresponsible. And generally not doing upgrades when believing them necessary, similar.

The “why” of Aave upgrades is a combination of all of those, that even if we tried to explain on the list of features, it is not straightforward without going even more technical.
However, maybe the two following points can explain better “why” upgrading Aave v3 makes sense:

1. On everything that relates to changing code aspects of the protocol for better patterns and security reasons, we firmly believe that the risk of not doing it and an unknown problem potentially affecting those is, even if not high, higher than the risk of hurting the system while upgrading it. Hence, the logical step is doing them.
2. The features more on the UX/dev-ex experience are what historically enable Aave v3 to be competitive. By not making any changes to those, Aave loses on its potential, and the whole community, stakeholders, and other service providers suffer the consequences.

As we previously commented, though, it is impossible to categorically say on all aspects “this is better, this is worse” on a system as advanced as Aave v3. And from the community perspective, evaluating a decision like that boils down to:

• Do we trust enough both procedures and the parties involved in the suggestion? If that’s not the case on upgrades, Aave has a very big organisational problem.
• Even not being technical, do the suggestions at least sound reasonable overall? On this, we tried our best, but we definitely we are more than happy to elaborate even more on every single point proposed.

And last but not least, there has not been any meaningful issue with any upgrade in the past of Aave v3, and even if not, because of that we “relax” anyhow (total opposite), that should be taken into account.

[Emilio]

@bgdlabs

We have a very firm stand that, knowing that, deploying protocols in an immutable manner is plain and simple irresponsible. And generally not doing upgrades when believing them necessary, similar.

this is a pretty strong and debatable opinion, to be honest. Personally, i see the value of upgradeability and you raise fair points (although these points are becoming less and less relevant as the technology matures - they were much stronger 3/4 years ago, things are settling much quicker today). I also think that immutability CAN and SHOULD be aimed for, if the technology allows for it. There are amazing examples of protocols that have been immutable since the beginning of times, uniswap is one (and while V1/V2 are very simple from a design standpoint, V3/V4 are rather complex yet they are immutable and operate flawlessly). Would you call the uniswap labs team irresponsible? I don’t think that’s fair to them, they are amazing at what they do. While Aave is a different beast in general, and some Aave competitors have attempted the fully immutable route (including configuration) with mild success, i think an approach with immutable contracts and mutable configuration would strike the right balance, if done properly. Aave V4 has been designed with that in mind, being much smaller and easier to audit and yet packing more features than V3.
Anyway this is a bit off topic so i will not pollute the thread, but this is definitely a discussion the DAO needs to have, because imo seeing immutability as a bug and not a feature of blockchain development can be detrimental in the long run.

@bgdlabs

• Do we trust enough both procedures and the parties involved in the suggestion? If that’s not the case on upgrades, Aave has a very big organisational problem.
• Even not being technical, do the suggestions at least sound reasonable overall? On this, we tried our best, but we definitely we are more than happy to elaborate even more on every single point proposed.

I don’t think these two points are even debatable, of course everyone here including me and others raising concern trust @bgdlabs and the proposed features sound definitely reasonable. Problem here is that even if i fully trust your development skills and security procedures, i am also realist and i know anyone can make mistakes, including me and you. Or maybe you can guarantee, with 100% certainty and past any shade of doubt, that it cannot happen and there will never be a mistake ever? If you can, hats off to you. Also, to your point “upgrades have been flawless until now” which is undeniably true, it’s also true that past performance is never guarantee of future results.

On the second point my concerns are

1. how big is the impact on the codebase?
2. Even if the features are good, and the risk is small, are they worth the even one in a hundred thousands chance of introducing a regression and losing 30b?

I can’t evaluate 1. at the moment, and the 2. quite frankly, to me feels like a no. Hence my doubt of being fully supportive of this upgrade.

Last thing on the broader topic of upgrades, i think i raised privately with @eboado already - the approach of upgrading all the markets at once without giving a production testrun on smaller ones always felt a bit far fetched. I think it would be safer for everyone if at least a subset of smaller, lower tvl markets were upgraded first and left in production for a few weeks before upgrading the biggest ones, in order to have slightly more guarantees that no regressions were introduced. If this doesn’t change i won’t be supportive of future upgrades, because i think it’s a small change in the deployment procedure that brings great additional security guarantees and i don’t see why it shouldnt be adopted.

[eboado]

Writing in this case as co-founder and one of the leaders of @bgdlabs, not only as an Aave community member.

The conversation has derailed from the topic at hand, which is not beneficial for the goal of this thread.
Therefore, I want to outline what we will do from BGD as the next steps, and reduce our participation to answer any questions regarding what is included in the proposal itself:

• Early next week, we will create an ARFC Snapshot vote for people to pre-approve v3.4, which is the same as usual. Important, this stage is not the activation of Aave v3.4 in production, but a soft approval for us to proceed with final review, security, and setup steps.
  The community can vote against for whatever reason, thats the point of a decentralised organisation. But a clear negative outcome will signal that we should not proceed at all with v3.4, also that the community simply doesn’t trust our base criteria on v3.4 being a good upgrade for Aave. Because let’s be clear, the ARFC stage (not the activation even) on a proposal by an engaged service provider, on a process that has been made multiple times in the past, is also a vote of trust on our work, or the opposite.

• If the vote is positive on Snapshot, we will proceed with the usual security and quality control procedures, engaging third-party auditors and/or organising security contests. Aside from Certora, which is engaged by the DAO, we choose the appropriate ones, negotiate the budget, and cover it before the AIP refund.
  During this period, we are happy for anybody with proper credentials, like the case of @Emilio, to review the code if desired, give feedback, or suggest anything of importance.
  This should be done obviously via appropriate channels like Github, as this forum is not the place.
  And similar to any third-party review, as code owners, we always decide what makes sense to include, what not, changes to do, and changes not to do. Needless to say, nothing will be moved to an on-chain proposal if any finding has any indication of creating problems, but us not having final decision power on our proposal is exactly like any other team developing a system → engaging some auditor, and the auditor dictating what should be finally done or not; it is nonsense, and will not happen as we simply can’t do our work like that.

• Finally, after all security procedures are cleared, we will proceed with an on-chain AIP. Code be 100% public before this stage, audit reports too, and the magic of a decentralized organisation is that people can just take their AAVE → call the governance contracts, and approve the factual upgrade, or not.

[Emilio]

I honestly don’t understand this reply. I don’t understand why a potential negative outcome automatically means your work is not trusted. I don’t understand why it’s so hard to accept that DAO members, users and stakeholders have the right to be concerned about something that is announced without prior knowledge that it was even happening. It’s my understanding that the early phases of a proposal are about collecting feedback, and this is exactly what is happening here.
Of course as service provider you have the right of suggesting what you believe is appropriate. Me and others as AAVE holders, users and community contributors, have the right to be vocal if we believe something you are doing requires feedback. I was happy with the 3.2 and 3.3 upgrades because besides being a no brainer from a feature standpoint, the changes were intuitively self contained and relatively easy to understand. In this one, the impact seems somewhat more widespread on the codebase, with a benefit that is much less clear compared to the previous upgrades (note that i said seems, because i can’t evaluate yet as the code isn’t public). You should be able to accept this as a constructive feedback rather than an automatic “there is not rust in our work”. In theory, blockchain should be about minimizing trust in the first place. We are talking about upgrading a system with 30b inside, where a single mistake can cost everything. I am sorry, but i am not gonna stay silent and automatically accept if i believe something requires more clarification, this has nothing to do with trusting your work.
My suggestion for the ARFC snapshot is to publish the code and give enough time to anyone who is interested to properly review before the vote, because otherwise it’s gonna be hard to take an informed decision.

[0xmonk]

Agree. There is no need to “hurry” with the upgrade. The top most priority is safy and trust of the protocol.

[stani]

It doesn’t make sense to vote after few days of preliminary discussions to “soft approve” something that could get turned down upon actual activation. That would be waste of DAO budgeted resources.

Because let’s be clear, the ARFC stage (not the activation even) on a proposal by an engaged service provider, on a process that has been made multiple times in the past, is also a vote of trust on our work, or the opposite.

The framing from @eboado here where the vote against unnecessary protocol upgrades is vote of trust is quite unfounded. Community members including a technical member raised the concern of protocol upgrades, which by itself is a controversial topic in DeFi and our industry widely. As I personally mentioned earlier it made sense to upgrade for V3.1 given the essential security improvements, V3.2 and V3.3. itself were justified (even though the procedure of upgrading could have been risk mitigated more by upgrading one network deployment at a time). However, the concerns are around the version V3.4. Especially since the upside is very negligible for users compared of the risk of mutating existing codebase and recompiling it. Aave is not a small protocol anymore and each time the codebase is changed, the lindy effect restarts since new code is introduced. This is not only the perception but also the reality.

This direction of frequent protocol updates is playing with fire. It’s just a question of time, a human error occurs, not when, and once it happens the consequences will be devastating given the amount of funds in the protocol (circa 30B). In fact, we should steer towards a direction to ensure maximal mitigations of unnecessary risk and single point of failures. Aave is well built brand and it can all be lost by one mistake even if we are extremely diligent as we usually are.

Whole point of decentralization is to create resilient systems where the users do not need to trust us as the developers or rely on anyone else to ensure functionality of the system. This is the right vision, otherwise Aave is just another centralized business with additional blockchain hurdles but thats not what we are here for or believe in.

I also don’t see a point of altering a codebase that is going to be deprecated overtime similar to V1 and V2 while taking unnecessary risks with limited time of development and due diligence. At this point Aave’s moat is security and we need to steer towards direction where we try to minimize any potential risk and single point of failures, including code changes. Every single line of code change can be an additional risk and should be questioned. Every single protocol upgrade needs to be questioned, the more value the protocol has the more resilient the community should be towards protocol upgrades as stakes and their consequences are increasingly high.

[eboado]

I think there has been some misunderstanding on how ARFC and pre-AIP procedures have worked and work.

If the community signals positively on ARFC, that doesn’t mean the code is 100% closed to changes. It means that no major feature will be added to the codebase that will be the content of the on-chain AIP.
But the following can (and does historically) happen during that post-ARFC → AIP period:

• We (BGD) notice that some part of one of the proposed features can be slightly better, and we adjust, but within the same feature.

• If any bug is noticed in v3 production and has any type of urgency, we include the fix in the release.

• From all the feedback of security/reviewing parties, it is very, very common that we apply some changes to the features included, or in some cases, remove some of them. This includes rollout procedures, for example.

The previous is precisely why it usually takes weeks from ARFC approval to on-chain AIP creation, and the same will happen here in v3.4.

And even if for any reason somebody would have a strong opinion against the final version of the AIP, the code is made public way in advance that (again, a refined version of what was presented here, maybe with equal of less features, but not with more), and the activation depends fully on Aave on-chain governance.

So my (and BGD’s) point is that if we can’t even start that process, then it is impossible to proceed.

[JavierM]

Guys, the debate goes nowhere.
Both sides has a point, but i personally need more data to decide.

@stani the original proposal of V4 stated Q1-Q2 the code will be ready, i understand timelines are hard to predict, but Q1 has ended. Are we still in time to go live in Q2? If not, what is a realistic time V4 will be up and running?
Also based on past migrations, @Emilio how much time will take for the capital to migrate? V2 still has around $1B and it’s successor is live for 3 years now.
If the answers to the questions are far enough into the future I’d say new features are beneficial based on @bgdlabs track record.
In any case there are security concerns i expect them to be patched.

[Emilio]

After discussing internally the technicalities of the upgrade with @eboado, i am temporarily supportive to move the ARFC forward and once i get access to the code i will be able to assess better and eventually contribute with constructive feedback, if needed. On a side note even though a bit off topic, i think this process needs to be revised because relying only on the onchain vote for such critical tasks like protocol upgrades can pose a security risk in the long term. I think community contributors should be able to thoroughly assess the changes with at least one offchain vote before the AIP process.

@JavierM

@stani the original proposal of V4 stated Q1-Q2 the code will be ready, i understand timelines are hard to predict, but Q1 has ended. Are we still in time to go live in Q2? If not, what is a realistic time V4 will be up and running?

quick answer not to pollute the thread, code is in good shape and we started introducing SPs (certora first for obvious reasons) to it. Certora actually suggested a great optimization of the risk premiums that required some rework, in any case the release process and introduction to the technology that @bgdlabs mentioned earlier will start soon, and before the go live everyone will be able to operate on V4. We should be on time for the planned release.

[Certora]

The concerns raised in the discussion are important, as every code upgrade inherently carries a degree of risk.
As a security provider for Aave, our role is to ensure that any deployed code is as mature and secure as possible.
We remain dedicated to executing our work to the highest standard, as in previous upgrades, in collaboration with the developing entities and in alignment with the DAO’s decisions, ensuring the code reaches production in the safest and most robust manner.

[bgdlabs]

We have created the ARFC Snapshot for the pre-approval of the Aave v3.4 candidate, with voting starting in approximately 24 hours.

Participate
https://snapshot.box/#/s:aave.eth/proposal/0xbbdd44ff07184dc17b9215414f5bb747a48c19e699c7505df35a7e1ca54e9da6

As extra clarification, this approval only confirms a candidate set of features and changes, whose scope can only be reduced in the AIP stage and not increased unless for bug fixing on production problems.

[stani]

@JavierM V2 has around 300M TVL, not 1B (in comparison its 1.7% of the TVL of V3)

Based on my recent discussion with @eboado and highlighting concerns around new production code changes and upgradability, I am comfortable to temporarily pre-approve the Aave V3.4.

Going forward @AaveLabs and @bgdlabs will propose a Upgradability Framework that will give certainty for users over in which circumstances production codebases can be upgraded and when upgradability cannot be applied, including solidified procedures of the process. A resources that can be helpful for users.