Updated Proposal: Chaos Labs - Risk & Simulation Platform

[omergoldberg]

Chaos Labs - Risk and Simulation Platform Proposal

Proposal Updates

We have received valuable feedback on the Chaos Labs <> AAVE engagement proposal. After conversations with multiple DAO members, we have adopted the feedback and revised our original proposal. The spirit of this proposal is to optimize Aave protocol ROI and give Chaos the opportunity to put its best foot forward in driving significant value to the community over the long-term. We are excited to begin contributing full-time and confident that the quality of work and speed of execution will speak for themselves.

Revised Scope

Chaos Labs will focus on risk management and parameter recommendations for all v3 markets. This includes:

• Risk parameter recommendations and tooling for the community covering:
  • Borrow & lending caps
  • Collateral factors
  • Interest rates
• Asset listing risk analytics & tooling
• Open-sourced agent access
• Extending the AAVE v3 Risk Analytics App
• Community Risk Calls

We have removed all work regarding GHO and reduced the v3 scope to be more clearly defined. Based on conversations with the community, the release of GHO has been postponed, and thus work from external parties is out of scope. We believe our platform can be beneficial to launching this product and hope to be involved as it gets closer to launch.

We have additionally broken the payment structure down into:

• $500k base fee
• $350k in incentive bonuses (total) based on platform delivery to the community outlined below

The full term is 12 months, but after an initial 6-month period, the Aave DAO will have the opportunity to terminate the relationship if it finds Chaos Labs’ deliverables unsatisfactory (outlined below).

---

Summary

Chaos Labs is proposing to onboard the Aave community into its risk and simulation platform to test Aave v3, new protocol upgrades, and parameter recommendations in various market structures and scenarios. This platform will support the community in onboarding new collateral types, assets, and bespoke protocol research with publicly available analysis and results.

Who is Chaos Labs?

Company background

Chaos Labs is a software company building a unified simulation platform that allows teams to test protocols efficiently while understanding how they will react to adversarial market environments. The backbone of our technology is a cloud-based, agent- and scenario-based simulation engine that allows users to create specific market environments to test new features & assets to understand risk parameters better. Our team comprises top engineers from companies such as Apple, Facebook, Instagram, Amazon, Microsoft, Google, and more with years of experience in infrastructure, security, and platform “chaos” engineering.

The Chaos Labs simulation platform and environment are built to be as close to mainnet as possible. Each simulation run forks from a specified block height (default block height is the most recent) so that your inputs include up-to-date account balances and the latest smart contracts and code deployed across DeFi. While testing volatile environments, it is imperative to look at your protocol holistically. The Chaos Simulation platform helps understand how external factors (cascading liquidations, oracle failure, gas fees, liquidity crises, etc.) will impact a protocol in various situations.

Company values

Our mission is to secure and optimize protocols through verifiable agent- and scenario-based simulations.

• The best simulation testing is as close to production as possible. The Chaos Labs’ cloud platform will spin up an EVM-compatible forked environment for every simulation. Since all simulations are executed on a fork, all code deployed to Chaos can immediately be transferred on-chain and/or used for production. An additional benefit is that the fork gives us a snapshot of mainnet out-of-the-box. This allows us to run simulations with minimal assumptions and deviations from mainnet conditions.
• Trust, but verify. We can build a test environment and convince you that it is correct, but that trust only goes so far. Our tooling allows community members (i.e., simulation creators) and anyone they permit (up to the entire community) to dig into the test environment and push back against assumptions, agent creation, scenario environments, optimization trade-offs, and more. Open source agents and scenarios allow the community to understand precisely how answers came to be determined. We have built a suite of tools and libraries for rich data visualizations, auto-generated analytics reports, and internal block explorer so that users can verify the simulation results down to each block and transaction.
• Community engagement. We know that we can build the best tools to help test and optimize DeFi protocols, but we can only scale with lasting change if the relevant contributors are on board and actively engaged. Each protocol differs from the code to community risk tolerance, and we do not want to be the only voice translating that into proposed changes. We are a software company building tools to understand and mitigate DeFi protocols’ risks, ideally powered by the communities who care most about them. Thus, we want to engage the said community in each proposal, from simulation creation to testing review to proposed change enforcement.

Why economic security and testing are important & how Chaos protects against it

Security audits and penetration testing are crucial parts of the security stack, but they alone are not all-encompassing to limit the surface areas of vulnerability. Their primary function is to ensure that your code does what you want your code to do and that there are no major flaws, assumptions, or errors in what you wrote as you wrote it. We view Economic security as the next piece of cheese in the security stack (ref: Swiss Cheese Model), building upon the correctness of their reviews and manipulating the environment around the protocol to ensure the intended behavior plays out as intended in different scenarios ranging from business as usual to black swan events.

Since the rise of “DeFi summer,” we’ve seen nefarious actors managing to manipulate core protocols in increasingly creative manners. They are no longer looking for flaws in code, but they are manipulating the market around the target protocol to gain entrance and exploit it. This roundabout attack vector heightens the need for complex parameter setting procedures; knowing how different values for certain assets react in different environments will allow for more confident governance and usage of the protocol.

Proposal

Over this engagement, we will deliver a suite of new products for contributor parameter analysis and open the tooling up to a broader community group to run it in the future. As we embark on product development with Aave, we anticipate managing simulation creation, feedback integration, and reporting until the platform is ready for community control. In the future, we can create a new AIP proposal creating an Aave-dedicated simulation creation and analytics team (similar to the Risk DAO proposal), which can provide another voice in risk-mitigation conversations powered by data.

To continue to enhance risk coverage of the Aave protocol and transparency to the community, we’d propose tooling to cover a few major areas:

• Risk parameters for all v3 markets (Borrow & lending caps, collateral factors, interest rates)
• Risk analytics and parameter tooling for the community (dashboards beyond just proposals)
• Asset listing risk analytics & tooling- Open-sourced agents access

Simulation engine platform & unified infrastructure

From a risk and infrastructure standpoint, we see a number of tools that need to be developed and maintained for Aave to increase its security coverage on top of that provided by teams like BGD, Certora, Gauntlet, & Sigma Prime. The current risk coverage covers Aave v2 (well) and asset onboarding (less so) but can be enhanced to analyze and optimize a number of areas with specific simulation and dashboard tooling to be delivered to the community.

Chaos Labs has developed a novel, cloud-based, agent- and scenario-based simulation platform. Our product is built on the ethos that a valuable testing environment is as close to a production environment as possible. Therefore, we utilize a hybrid approach of on-chain and off-chain simulations.

On-Chain Simulations

On-Chain Simulations fork the blockchain from a specified block height and deploy a catalog of agents, scenarios, and observations within the Chaos Cloud environment.

Agents emulate user behavior and allow us to emulate different risk behavior for protocol users. The Chaos Scenario Catalog lets us control macro variables and conditions such as gas fees, DEX and protocol liquidity, oracle return values, Black Thursday Level market events, and more. Observers allow for deep protocol analysis and better simulation insights.

Through this robust software, users can control and test a host of different factors that can impact protocol security and user funds, including

• Oracle prices
• Gas fees
• Account balances & liquidation prices
• Transaction latency
• Flash loans

Economic security testing and simulations via the Chaos Labs platform allow you to test your protocol in different scenarios and custom environments to understand where your risks lie before a malicious actor can exploit them. Some examples:

• volatile markets impact on protocol reserves
• asset correlations and liquidations
• the effect of drastic price drops on liquidations and liquidity
• the impact of high gas fees on the efficiency of the liquidation process
• new asset borrow demand, revenue, and liquidation processes

In this manner, we will integrate directly with the Aave protocol and provide transparent simulation insights.

Off-Chain Simulations

Chaos Labs also deploys off-chain simulations, utilizing machine learning and statistical models that ingest data sets from various off-chain data sources to test economic structures prior to any solidity or on-chain code being written. As part of the off-chain simulations, Chaos Labs will run a massive number of Monte Carlo simulations to assess the protocol’s VaR per Market (Chain) and across markets.

A combination of On-Chain and Off-Chain simulations allows us to control and test a host of different factors that can impact protocol security and user funds including:

• Oracle behavior
• Gas fees
• Account balances & liquidation prices
• Transaction latency
• Flash loans
• volatile markets impact on protocol reserves
• asset correlations and liquidations
• drastic price drops impact on liquidations and liquidity
• high gas fees impact the efficiency of the liquidation process
• new asset borrow demand, revenue, and liquidation processes

Product Screenshots

Asset Listing Portal

One of the focuses of this engagement would be building an Asset Listing Portal to help streamline new collateral onboarding to the Aave protocol similar to what we have built for dYdX, found here. This tool will help streamline community decision-making by automating the collection and analysis of key markets data around assets such as:

• Market beta & volatility
• Exchange liquidity and slippage
• Market cap
• On-chain activity
• Third-party lending integrations (i.e. compound, maker, etc.)
• Security & demand scoring
• Revenue estimation
• Initial parameter recommendations

This tool will help streamline the addition of new assets to the Aave platform, thus increasing platform fees to the treasury and token suppliers while balancing the overall protocol health.

Community engagement

Community Risk Calls

As part of our commitment and efforts towards community engagement to further drive protocol security, Chaos Labs would organize a monthly risk call for the Aave community alongside DAO contributors. This call would be focused on any new major protocol or market developments such as:

• New risk tooling and analyses
• Protocol launches and technical proposals (GHO, v3, etc.)
• Asset listing proposals
• The broader market environment
• and anything else the community deems important and relevant for discussion

We would schedule said calls for a recurring hour-long block on a monthly basis in addition to any ad-hoc community risk calls when deemed necessary. A recording and summary of these calls will be provided.

Ongoing updates

Aave’s dedicated relationship manager will be an active participant in organizing the risk conversation and updating the community in the forums. We will commit to a monthly update post focusing on both works complete and ongoing as determined by the community. We will also host monthly office hours to be available for community Q&A.

Long-term relationship

As has been stated above, we are a software company at our core. We’re building a robust platform that empowers communities to develop, test, and risk-manage their protocols at a more sophisticated level without needing to rely on any single outside third party. While our focus is to use this engagement period on product development for the Aave community, our hope is to eventually onboard a consortium of community members to create the relevant testing environments and risk evaluations for the Aave protocol on top of the Chaos Labs platform. We promise to be as transparent as possible during the process while it is centrally managed to build towards this more open and decentralized future.

Measures of Success

Security and testing is a tough realm to measure appropriately. The successful completion of the Aave protocol’s objectives will be measured against KPIs that will be derived from the specific objectives agreed upon between Aave and Chaos Labs. On top of those, We will also look to measure things such as:

• Losses incurred in covered markets (or lack thereof)
• Product deliverables (Asset Listing Portal & community parameter setting tools)
• Community NPS of our relationship
• Communication and transparency to the community on work done and product access

Previous Aave Work

• Aave v3 Risk Application
• Diving Deep into Aave v3 Subgraph Data Validity
• stETH Depeg Simulation and Analysis
• stETH Depeg Simulation Video Walkthrough

Pricing

• 12-month engagement term

  • Note that Aave DAO has the ability to cancel Chaos’ contract and stream at the 6-month anniversary of the public vote. We will provide a detailed report on our progress that will provide the community with sufficient information to make that decision.

• $500,000 flat engagement fee paid in USDC streamed linearly over the course of the contract

• $175,000 paid in AAVE tokens payable upon delivery of the Aave Asset Listing Portal (7-day TWAP)

• $175,000 paid in AAVE tokens payable upon delivery of the Aave Parameter Recommendations Tools (7-day TWAP)

  (Delivery is defined by open access of the tool to the community shared in the Aave Forum)

[AaveLabs]

We are pleased to see active community engagement throughout the evolution of this proposal and continue to be excited by the developing contributor landscape focused on risk for the Aave Protocol.

Overall, strong risk management is central to the protocol’s success, and it is positive to see multiple contributors in this area take part in Aave’s governance process and show an interest in providing services to the DAO.

[lucasvo]

I’m supportive of the adjusted scope and am excited to see this proposal going for a vote.

[Alok_StandardCrypto]

Disclosure: Standard Crypto is an investor in both Aave and Gauntlet.

We’re glad to see market risk getting more attention recently. While the merge and concerns regarding stETH have passed, the development cadence and complexity of Aave will continue to make risk management important. The Aave community needs to continue our leadership on this front.

Regarding the proposal, we believe simulation has an important role to play for risk management. We’d love to better understand the details of how this approach adds to the suite of risk-related services the DAO already consumes.

@omergoldberg - Would you be willing to go into a bit more detail on your approach and how it complements what others (e.g. Gauntlet) are currently doing for Aave?

It would help us, and others I’m sure, form a view when it comes to a formal proposal.

[omergoldberg]

Thank you all for the feedback and support. It’s encouraging to see how deeply the community values and prioritizes risk management. We’re excited to begin contributing full-time and driving impact for the Aave community.

Regarding our approach to simulations - we’ve provided documentation, blog posts, and video tutorials which can be found throughout the proposal. We welcome you to review those as they will provide further detail about the Chaos platform. As the DAO matures and continues to lead the industry in Risk Management we are supportive of a multi-risk vendor approach which is common practice in traditional finance.

The product we will provide for Aave v3 is complementary to existing risk partners. As mentioned in our prior post, v3 has not received support since launch, despite billions of dollars worth of assets across borrow and supply markets. v3, along with other non-v2 markets was out of the scope of previous Gauntlet engagements.

We’ve also addressed any potential areas of overlap previously:

@omergoldberg

Areas of Overlap

• For areas of overlap, our opinion is that multiple risk vendors are a feature, not a bug. AAVE is a best-in-class borrowing protocol, currently managing 10 billion in TVL. The benefits of robust risk management cannot be overstated. The DAO can benefit from multiple vendors conducting independent and collaborative research. Underlying differences in technology will result in analysis drift and varied results and recommendations. This will allow the community to compare results, conduct discussions, and ultimately make informed decisions. We look forward to facilitating and driving these discussions on the forums (async) in conjunction with the monthly community risk calls proposed above (a specific ask from community conversations about this proposal). Additionally, relying on a single vendor creates a clear centralization risk. A single bug or misjudgment in recommendations can have catastrophic consequences. In traditional risk and finance, the best practice is to source multiple vendors with different methodologies and risk assessments. These are then consolidated and referenced to drive discussions and ensure that there is a holistic analysis.

I can’t speak to the exact differences between our product and Gauntlets since the Gauntlet platform is closed. The available literature is high-level and does not delve into implementation details. Obviously, in agent-based modeling, every agent feature, parameter weight, price trajectory, slippage model and more have a great effect on the end result. As the models, weighting, and execution environment (our simulations run on blockchain forks) are different we will expect a drift in results. For any future markets with overlap, the differences in results are great. They provide an additional data point for the community and will enable a data-driven decision-making framework for setting risk parameters. At Chaos, transparency and explainability are key company values: once we are engaged by the community, we will be open-sourcing our Aave-related agents for full visibility, review, and enhancement. We’d love for Gauntlet and other risk contributors to do the same. We’re excited to collaborate with all contributors and excited to push the standard of risk management higher.

While this proposal has complementary aspects (we are focusing on unserviced markets with new challenges such as e-mode, isolation mode, portals, and asset listing), I want to be clear: we are competing with Gauntlet (and other risk vendors) to build the best product possible and win Aave’s business. This competition should push all vendors to deliver the highest quality work which will greatly benefit the DAO. We believe that the current technology and products built for DeFi are primitive, and can be improved at least 10x. We’re excited to let our products and offerings speak for themselves.

@Alok - Thank you for disclosing the clear conflict of interest ahead of the vote with Standard Crypto’s and your personal early-stage investments in Gauntlet. We appreciate the transparency and believe it is key to the long-term development, sustainability, and advancement of Aave.

We’re appreciative of all the community engagement received over the past month with this proposal. With the positive feedback we’ve received on the revised copy, we plan to post a Snapshot vote tomorrow for community consideration.

The Chaos team is excited to contribute to the Aave DAO, and can’t wait to get started!

[tarun]

Thanks for bringing this up, @Alok_StandardCrypto. One major tenet of risk management, both in traditional finance and DeFi, is a notion of statistical rigor when choosing high value parameters. While we agree that there are many benefits to multiple risk assessments (which usually involve methodologies that can be aggregated in a statistically coherent manner), there seems to be little to no description of either a coherent statistical methodology in this proposal nor any level of rigor befitting billions of dollars of AUM in the case studies provided. Let’s perform a pedagogical understanding of why this is harmful and we would argue that any additional risk vendor should be subject to providing a real risk methodology rather than summary statistics.

Risk Analysis methodology shares similarities with Smart Contract security

The Aave DAO has employed a number of continuous auditing entities, such as Certora, to provide formal verification services. A number of formal verification tools (such as the K framework and Slither) exist so one may naturally ask the question, “Why do we need to pay Certora? Can’t we provide bounties to developers to write the formal verification code?” The answer is that it is extremely hard to construct a comprehensive set of tests and missing one can be deadly.

Let’s see why with a simple example.

There can technically be an unbounded set of properties that one needs to prove for a contract to be safe under all circumstances (especially if off-chain interaction is involved [0]). Suppose that there exists a set of 10,000 properties (or invariants) that a smart contract needs to satisfy in order for it to provide the expected operation under all expected circumstances. Furthermore, suppose a new developer who didn’t know that this was the requisite universe of properties and instead constructed a set of 100 properties that they were able to prove held true in the contracts using an open source tool. This developer might then publish these tests, add them to a continuous integration tool chain, and lead everyone to believe that the contracts were safe in spite of not actually having full coverage. The reason one employs OpenZeppelin or Certora is because they have a better sense of the universe of possible properties and tests for a formal verifier and can ensure that their committed properties (including those found via contest) are closer to covering 10,000 than 100.

Agent-based simulation (ABS) is in a similar spot to formal verification — it is a tool that only operates as well as its inputs are good. Like formal verification, there is a set of properties or expected behaviors that are tested based on the definition of the agents in the system and the context and/or environment for evolving these agents. However, unlike formal verification, the statistical assumptions made dramatically impact the output quality of the results. For instance, one may naively think that a high liquidation volume on a particular collateral asset, borrowed asset pair is a sign that a protocol parameter needs to be changed. However, liquidations are a healthy part of any leverage system and often, liquidations are anti-correlated with the worst type of behavior in the system: insolvencies. As such, an agent-based simulation that changes borrower behavior based on liquidations as a signal instead of something that constrains liquidation volume by insolvencies would deliver completely inaccurate parameter recommendations (much like covering only 100 tests instead of 10,000).

The original post’s linked case studies make many mistakes of this form, suggesting that the authors are treating ABS as a form of directed fuzzing rather than a statistically accurate, Bayesian learning tool. Some of the pieces of the case study that provide a severe lack of confidence in the statistical significance of the results include but are not limited to:

1. The authors depeg stETH to a large deviation and measure whether liquidations occur as a metric of protocol health. This misses a few major statistical observations that need to be constantly retrained based on off-chain and on-chain data:
   a. Such a price level is usually unsustainable if other venues have enough liquidity and arbitrageurs can profit from converging prices. This model seems to ignore this effect and the volatility (or any higher order moments of the price process).
   b. There is no description of the liquidity model used. Liquidation profitability is extremely sensitive to the liquidity on both off-chain and on-chain venues and these have different empirical elasticities due to flash loans making on-chain liquidity cheaper for risk-averse liquidators. We demonstrate all of these effects in our Aave market risk assessment from 2020 and urge the original poster (or anyone who wants to do risk for a large protocol) to read this carefully. This post suggests that they clearly did not.
   c. There is no stability analysis. The impact of such a shock on the system depends on volatility conditions throughout DeFi — the smooth curves rendered in the case study assume purely deterministic behavior which is very much not what you see on-chain or in the mempool.
   d. The assumptions are wholly unrealistic:
   1. “In this simulation, we ignore the effect of stETH de-peg on other asset prices.”
   - This is completely not true in practice and moreover, the liquidity and/or slippage curves of other assets are also correlated to stETH/ETH!
   2. ”We do not simulate any stETH buy pressure on Curve in order to speed up the cascading liquidation effect.”
   - This was in fact what ensured that Aave was safe — the original posters clearly did not look at on-chain data during the large liquidation events.

2. The simulations run for a period of 150 blocks which is too short to realize realistic arbitrageur behavior
   a. The actual stETH depeg took place over a much longer time frame and liquidity conditions changed dramatically within that interval. For reference, Gauntlet runs simulations for a minimum of 1 day, which simulates 5760 blocks and we run over 40,000 simulations per day sampling different statistical configurations. It is much harder to get convergence and confidence intervals when you’re running 0.00000065x the number of simulations.
   b. The authors seem to not understand that making an infinite CEX liquidity assumption and only running for 150 blocks implicitly chooses a volatility (which they don’t specify to the reader!). Choosing a stopping time for a simulation implicitly impacts the statistical quality of the results.

In general, the above mistakes make someone with even a barely trained eye have to question the conclusions and parameter choices made. While the original posters’ dashboards are a good way to visualize high level statistics, their inferences get so many basic things incorrect in a way that dramatically impacts parameter recommendations so much that one should be nervous of using them. Access to data sources is not the same as being able to derived insights from that data that you have immense confidence in.

High value parameters are hard to “throw machine learning at blindly”

At Gauntlet, we’ve pride ourselves on our two main contributions to the DeFi space: developing the core research needed for constructing good models of DeFi protocols [0] and ensuring that our continuously retrained and optimized models match real life predictions. We spend much more time evaluating the out-of-sample complexity error of our parameter recommendations than we do make new agent logic. And there are a number of good reasons for this:

1. If you can predict out-of-sample insolvencies with a high precision-recall curve, then you can be confident that your parameters match an optimal value
2. Being able to test deviations from no-arbitrage assumptions allows one to improve liquidity models throughout DeFi (which in turn, impacts liquidators’ realized costs, which is the most important part of the economic model)
3. Understanding the trade-off between prediction quality for liquidations, revenue generated, and insolvencies and how the trade-off changes as the Aave loan book evolves is crucial to being able to automatically submit recommendations.

We run over 300,000 simulations of the Aave protocol a week, with an eye towards how model quality changes when we use a large-scale hyperparameter optimization algorithm for choosing financially important values like loan-to-value. Being able to measure model quality relative to realized behavior on-chain is one of the most beautiful things about DeFi — in traditional finance you wouldn’t have the entire data set at your fingertips. Model quality in DeFi is equal to dollars earned for tokenholders and lenders.

Unfortunately, the OP seems to not understand this. Other hand risk management entities in DeFi, such as Risk DAO and Block Analitica, which differ in methodology and precision requirements to us clearly understand this fact based on their track record of published research (which Chaos appears to have none of, especially to the caliber expected of a protocol managing billions in assets).

Finally, a prior version of OP’s proposal mentioned that they would compute “VaR” without describing any methodology for doing so. We’ve written a number of articles, from our Aave market risk report to our description of VaR and LaR calculations that provide a precise methodology for computing these values. Chaos seems to not understand that VaR is sensitive to one’s distributional assumptions in a simulation. After all, VaR is simply defined as a probabilistic tail bound. But how do you compute that tail? The empirical cumulative distribution function depends not only on your agents’ logic but on the statistical assumptions you make of the environment (prices, liquidity, other protocol behavior) and needs to be constrained by that. Having a risk provider who doesn’t understand such basic facts (especially when there are at least three others who do!) seems malevolent towards users of the protocol.

Finally, one other major flaw in the OP’s post is that they choose some protocol parameters and not others. In particular, they clearly miss all of the liquidation parameters which are obviously crucial to incentivizing safe operation of the protocol. Moreover, the OP doesn’t seem to understand that when you optimize LTVs, you need to do this jointly (e.g. optimizing the n² LTV parameters and the O(n) liquidation parameters) simultaneously. If you don’t, you run into an issue where you have optimized for an LTV that doesn’t cause insolvencies but only at an unrealistic liquidation threshold (e.g. 50%, which would be unacceptable to users). Such an oversight suggests the OP hasn’t even thought about how to do the task they’re asking $500,000 for!

Track record

Much as formal verification companies earn a track record by proving that they are able to cover the space of properties well, so too should risk assessors. We have one of the longest on-chain track records for providing risk recommendations to protocols throughout DeFi. Moreover, we’re even putting money on the line for it and putting our money where our mouth is. When making risk parameter changes, you are making changes that impact billions of dollars of AUM and user welfare — we don’t take that lightly and are committed to putting up capital to cover any losses that are inured from bad predictions. The OP seems to not recognize this, which perhaps explains their flippant modeling style. We note that this was also brought up during @bgdlabs’s comments on the original proposal (and we note that this proposal does not elide all of BGD’s prior considerations).

We note that we take integrations (and the statistical reliability of our simulations — which are 7 orders of magnitude per day more than what OP seems comfortable using to manage billions of dollars) seriously and make sure that our model quality is up to the highest standards before we support a new version of a protocol. Given the careful rollout of Aave V3, we’ve spent an enormous amount of effort making sure that we model the liquidation mechanics and liquidity impact of e-mode in a manner that is of the highest quality (as we of course care about our track record).

Conclusion

Decentralization brings a lot of really amazing things to the risk management community. Firstly, the free access to data from every protocol and liquidity source allows for one to create rich models of user behavior using tools such as agent-based modeling. Moreover, many entities can develop their own tools and insights on this data and the aggregate can be better than the whole. These tools and aggregative insight is the thing actuaries in traditional finance only dream of! But at the same time, one can only extract as much from their tooling as the inputs they put in. As they say in statistics and machine learning, “garbage in, garbage out.”

The above proposal seems to be more of a proposal for setting parameters via fuzzing. The lack of care in the prior art, seemingly large omissions, and dramatic changes from the last proposal do not inspire confidence. Given that there are a number of other organizations, Gauntlet included, who are much more rigorous about this, it behooves tokenholders to be careful when considering such a proposal.

If the goal of Aave tokenholders and @AaveLabs is to help grow the protocol, bring in institutional capital flows (via mechanisms such as Aave Arc), and take advantage of the benefits of DeFi, then we need to have risk management that can be trusted. Institutions demand a high level of rigor in risk management, especially with regards to statistical rigor and a careful calibration of predictions to realized outcomes. To get DeFi to that level, we as a community need to continue to invest in building better risk methodologies and creating research and tools for the community to understand the inherently complex properties of risk in DeFi. This proposal achieves none of that, making it impossible for us to support nor recommend anyone else support it either.

---

[0] In fact, the name ‘oracle’ derives itself from referring to an arbitrary Turing machine providing you with the output on given input. This, of course, includes the halting problem and hence you have a technically unbounded state space

[1] We are coauthors of over 24 papers in DeFi, including the highest cited paper within DeFi. We’ve also written numerous papers on how to analyze lending protocols, including modeling how LTVs/collateral factors should change when one borrows capital against an LP share.

[lucasvo]

I think Chaos Labs will be a nice addition to Gauntlet’s risk work. Specifically what I’m excited about is the fact that the tooling Chaos Labs created allows people to analyze and create their own view on the state of the system. It’s a software solution that over time will only get more powerful. Software alone doesn’t replace the need for expert analysis and researchers but I think it’s a good addition and has the potential to make this analysis faster.

I really like the fact that this tech is open for anyone to use and increases transparency and accessibility of risk management within Aave.

As far as I understand this proposal isn’t suggesting to replace Gauntlet’s work with what Chaos Labs wants to do. I don’t quite understand the strong opposition from you @tarun here?

[DiamondRock]

JD here, CEO at Benqi, a borrowing protocol on Avalanche. First of all to be clear - we don’t have a stake in Gauntlet or Chaos Labs. Similar to what is proposed here, we are currently customers of both and have worked closely with Chaos Labs over the past year across various market conditions and initiatives. We think both firms are great - however, I’ll focus on Chaos Labs since they are the ones being discussed here.

Benqi takes risk very seriously, which is why we use a multi-risk vendor approach. While the Aave DAO will ultimately decide what’s best, we felt important to share our experience, which has been amazingly positive with Chaos Labs. The Chaos team is excellent, honest, and hard-working. In my opinion, they are world class and have guided Benqi in times of high market volatility with top-notch risk tooling and simulations. Their speed of execution and quality of deliverables has been very impressive. We can attest to the degree of rigor Chaos applies to recommendations and could not agree more with the description of their competencies described here. Benqi’s safety is largely in part to Chaos’s contributions across not only lending market risk alongside Gauntlet, but also various other fronts such as liquid staking and the veQI launch. The crazy market downturn and market deleveraging were particularly challenging to navigate. Chaos was there at every turn of the way. We know they’ll be an asset and a strong contributor to any community they join.

[omergoldberg]

TL;DR:

• This post was an attempt to discredit Chaos and ensure vendor-lock in from a well-connected competitor
• The critique misrepresented a proof-of-concept video with attacks clearly stated in the accompanying disclaimers, but we have explained the relevant methodology accordingly
• We want to shift the entire risk of this engagement onto our ability to deliver, so we will work for FREE for the first 6 months of this engagement.

---

I have read this post multiple times and have seriously weighed the appropriate response. Tarun’s response is malicious, misleading, unprecedented, and unnecessary. It requires a clear, direct response. Tarun’s goal is to discredit Chaos and insinuate that we are unqualified to provide services to the DAO. This wasn’t only an attack on me, the CEO, but the entire Chaos team who has worked tirelessly to build a scalable platform to best support the DAOs who put their trust in us. I felt the need to respond in kind to put my support behind the 20+ people working at Chaos to make DeFi more robust and better secure individual user assets.

In this response, I will share why Tarun’s response is clearly incorrect, why Chaos Labs can and will deliver world-class services to the Aave DAO, then refute claims lobbed at us, and propose a revised path forward.

To get to the brass tacks: our initial proposal has been up for nearly 30 days without a single response or question from Gauntlet. During that time, we’ve received numerous messages that Gauntlet has been lobbying against us to prevent us from servicing the DAO as their potential contract renewal date approaches. While it was disheartening to hear about a competitor playing dirty and relying on clout/relationships to advance their business agenda, we were happy to focus on deliverables: showcasing the platform’s ability and ultimately letting the quality of our product and work speak for itself.

While many DAOs we speak with are concerned with “vendor lock-in” when choosing partners, they mostly look at it from a technical perspective. The back-door lobbying and gate-keeping from early DAO service providers is a far more concerning aspect of this entanglement and one we are now watching play out. These early participants, like Gauntlet, have the sufficient cachet to suffocate any potential competitors away from the DAO and prevent it from receiving competing services.

As an early member and contributor of this protocol, Gauntlet has both the connections and the token allocation to make a significant impact on how the community interprets major decisions. We are happy to engage on the merits of the proposal or those of the simulation platform, but not get into mudslinging across a proof of concept prior to engagement.

We also understand that stakeholders and community members may not be up to date with the inside baseball and tactics that Gauntlet deploys, so we’ll take the time to explicitly provide a pedagogical understanding of why Gauntlet claims are unbounded in truth, inappropriate, and just plain wrong.

Technical responses

The heart of the matter is that you’re attacking a single simulation that was built as a demonstration for the Aave community. Creating high-fidelity simulations and models can take weeks or months. You need to look no further than Gauntlet’s outline for v3 support, in which it references the “several months” it has spent to expand risk modeling to support v3.

In addition to the financial resources needed, training predictive models take time and engineering hours. This simulation is decidedly smaller in scope as a proof of concept as the real thing costs resources we can’t spare before the guarantee of an engagement, as should be expected from any vendor.

There’s a clear reason why we build a smaller scoped proof of concept. Running simulations is a resource-intensive task. The compute cycles involved in running them are not free. The time engineers spend running them is not free. The purpose of these POCs is to illustrate what the platform is capable of, not to stress test it. When you go to buy a car, you can take it for a test drive. You can see how fast it goes from 0 to 60, but you don’t get to run it for 300k miles.

The POC clearly demonstrates what our product can do, and mass-scale simulations will be run once there is a contract in place to handle the material operating expense involved. Suggesting that any service provider approaching the DAO should only do so after incurring an incredibly high expense is to introduce a form of bureaucratic capture and incumbency bias not dissimilar from what we see when big banks or big tech companies lobby congress to regulate their industry.

The work we have done for Aave, beyond the Risk Dashboard for which we received a small grant, has been well received. We built dashboards at half the price other grantees received for similar tasks. We spent days debugging Aave subgraphs, finding critical errors, and shipping PRs to benefit the community. We have been working with the community in good faith to show progress, create walkthrough educational materials, and demonstrate how our platform can be utilized, but at a fraction of the cost.

The fact that you are attacking and analyzing a proof-of-concept video is misleading.

Let’s clarify the purpose of the PoC and then address the claims lobbed at us:

First of all - the goals of the stETH:ETH depeg simulation were to:

1. Show the capabilities of the platform, both in how it interprets on-chain data and the transparency-focused tools built to allow for community analysis of the underlying simulation.
2. Highlight the minimized need for contract/protocol reconstruction and assumptions by not copying the functionality into a different domain-specific language.

At no point did we claim that the results of a single simulation are statistically significant. As such, all assumptions were clearly stated in the video and blog post.

Now let’s discuss the accusations.

Let’s begin by lumping all the attempts to confuse readers into a single bucket:

• “The assumptions are wholly unrealistic:

  1. “In this simulation, we ignore the effect of stETH de-peg on other asset prices.” This is completely not true in practice and moreover, the liquidity and/or slippage curves of other assets are also correlated to stETH/ETH!

  2. “We do not simulate any stETH buy pressure on Curve in order to speed up the cascading liquidation effect.” This was in fact what ensured that Aave was safe — the original posters clearly did not look at on-chain data during the large liquidation events.

• “Choosing a stopping time for a simulation implicitly impacts the statistical quality of the results.”

• “For reference, Gauntlet runs simulations for a minimum of 1 day, which simulates 5760 blocks and we run over 40,000 simulations per day sampling different statistical configurations.”

Again - It’s a proof-of-concept, not a case study or report. This is why they are stated as disclaimers and assumptions on the blog and video walkthroughs. Therefore, all these arguments are baseless.

Now I will address several points on which we feel the community deserves clarity.

1. “The authors depeg stETH to a large deviation and measure whether liquidations occur as a metric of protocol health. This misses a few major statistical observations that need to be constantly retrained based on off-chain and on-chain data”:

   1. “This model seems to ignore this effect and the volatility (or any higher order moments of the price process).”

      In the demo simulation, we examine an edge case of stETH de-peg at a time of block 15006921. At that time, stETH liquidity was primarily on Curve. We clearly address that fact, explaining the reasoning for our liquidity model:

      “Unlike most major crypto assets, where significant liquidity is found on CeFi venues, stETH liquidity is mainly provided on Curve. This allows us to simulate inter-protocol dependencies and effects with minimal off-chain assumptions, accelerating cascading liquidations.” (Sources: Link, Link)

      The model does not ignore arbitragers’ effect on price stabilization but utilizes the fact the stETH liquidity was mostly on Curve at the time of recording. Therefore, this claim is, again, meaningless. It is important to note that this was a conscious decision, not a technical limitation. We’ve modeled a variety of arbitragers into the platform for different simulations and will be open-sourcing their code for the community to review as we engage with Aave and release the parameter optimization platform publicly. Similar to other open-source development (i.e. subgraphs), it is our hope that multiple community members review these and help optimize them both for Chaos’ simulations and individual developer testing.

   2. “There is no description of the liquidity model used.”

      Chaos Labs simulations utilize a mainnet fork as our simulation runtime environment. As such, our agents are interacting with the Curve contracts as they would on mainnet without having to make assumptions on the liquidity model. Since the majority of stETH was on Curve during the recording, we believe this is the most accurate liquidity/market impact model for that simulation. Because of this, we can estimate on-chain liquidators’ profits more accurately.

   3. “The impact of such a shock on the system depends on volatility conditions throughout DeFi — the smooth curves rendered in the case study assume purely deterministic behavior which is very much not what you see on-chain or in the mempool.”

      The impact of arbitrageurs is addressed. We do make an assumption of rational, efficient liquidation bots. We believe that this is a fair assumption since although their execution time and order are not deterministic, their behavior is predictable as rational agents.

   4. “Choosing a stopping time for a simulation implicitly impacts the statistical quality of the results.”
      Nowhere in the proof-of-concept is there an assumption of infinite ****CEX liquidity. On the contrary, the lack of CEX liquidity provides the reasoning for looking at Curve Pool liquidity as the primary liquidity model. That fact enhances the effect of cascading liquidations, as shown in that simulation, and it is the reason we chose that edge case.

Team Background

As I mentioned, this post was not just an attack on my competency, but on the entire team at Chaos Labs. A team of hard-working people who want to see Aave and DeFi as a whole succeed. We’re not scared of DAO politics or incumbent bullying. We want to make sure that the community knows what it’s getting when engaging with us.

We are aligned with Gauntlet that the appropriate methodology to determine optimal parameters for DeFi protocols is Monte Carlo simulations at scale, but approach it from a different perspective. Our team has years of experience in building data-driven simulations, determining billion-dollar outcomes across FAANG companies, and matters of national security. I’ve personally led statistical experiments and analyses across internet.org, Instagram, and Facebook. Our team’s past experience includes building simulation software for calculating incoming missile trajectories and diagnosing and predicting medical diseases. We are well equipped to handle the challenges facing the DAO (and building these simulations at scale) and are eager to prove it throughout this engagement.

Proposal Pricing Update

As we clearly stated above - we will let our product and work speak for themselves and we are updating the payment structure to even further demonstrate this. We want to shift the entire risk of this engagement onto our ability to deliver.

We will work for FREE for the first 6 months of this engagement. Chaos does not get paid until after the 6-month anniversary when the DAO has the ability to terminate the contract. That means that if the DAO terminates, it will have paid Chaos $0 for work done. We will provide a detailed report on our progress that will give the community sufficient information to make that decision.

Compensation Model:

• $500,000 flat engagement fee paid in USDC streamed linearly starting at the 6-month anniversary of the public vote and streamed over the remainder of the contract
• Incentives based on delivery, payable no earlier than 6-month anniversary and based on a trailing 7-day TWAP:

  • $175,000 paid in AAVE based on delivery of the Aave Asset Listing Portal

  • $175,000 paid in AAVE based on delivery of the Aave Parameter Recommendations Tools

    (Delivery is defined by open access of the tool to the community and shared in the Aave Forum)

Conflict of Interest

We would like to highlight to the Aave community the inherent conflict of interest present between all parties involved here (Chaos, Gauntlet, Standard Crypto, and each side’s relevant investors) regarding the onboarding of Chaos Labs as a contributor to Aave. This proposal does not replace Gauntlet and we hope to have a productive relationship will all Aave contributors. With that said, we acknowledge that we are 100% competitive with each other in hopes of providing the best product and service to the protocol at the most attractive price. With token holder approval, Chaos Labs will prove that there are other methods — ones more transparent, efficient, and scalable — to properly protect Aave and that we are committed to helping retain treasury funds (and hopefully increase them) through the current market cycle.

Due to this conflict, Chaos Labs will not vote on this proposal and has requested its investors to do the same. We wish to allow the Aave community to independently determine the value of bringing us on board with the revised terms.

Conclusion

The amount of AUM at stake affects the magnitude and seriousness of the situation, but not the complexity. At the end of the day, this is an optimization problem that uses probabilistic predictions to find a balance between capital efficiency and economic security. The Chaos team is experienced and has a long history of working on complex data-rich, security-oriented issues in both the public and private sectors. We wrote the proposal to communicate it to the community in terms that are digestible to stakeholders and not to overcomplicate it with unnecessary technical jargon (which we have expanded upon above). In short, analysis paralysis is not the path forward.

The way we like to do business at Chaos is simple and honest. Ultimately, what we propose to do is straightforward. We’re offering a product and service that Aave needs at a fraction of the cost it’s paying for now.

We can go back and forth with an academic debate on simulation optimization models but would prefer to build first and debate once the community has access to our platform, models, agents, and results.

Fundamentally we believe that the long-term success of the DAO model is predicated on a diverse ecosystem of contributors and vendors. Such an ecosystem should create healthy competition in product quality and contract terms ultimately benefiting the DAO and maximizing the ROI. The goal of this revision is to remove any concerns of working with Chaos and make the decision to engage as easy as possible with no additional burden on the treasury required before delivering.

[tarun]

Making tooling to make it easier for people to understand complexity is clearly the commonality between Chaos Labs and Gauntlet. However, there are a few points that I made above that seem to be elided in your reply:

1. Software that is used to make statistical predictions (especially for risk parameters) has a ‘garbage in, garbage out’ problem — if I could verify that the inputs and the models were trained correctly and with some notion of statistical accuracy, then I could have confidence. Otherwise, I am at the whim of how good the author of the model is (analogous to my point regarding coverage in formal verification)

   a. The only way, without advanced cryptography, to measure quality is via a contest for providing recommendations (and we will have some news about something we’ve been working on for this soon™)

2. Chaos is proposing making active risk parameter recommendations which is what Gauntlet already does

   a. As described in detail in my response above, the lack of rigor and care in modeling that Chaos has demonstrated in their case studies would not really befit a protocol with billions in AUM. Other vendors, including Risk DAO, Block Analitica, and Gauntlet have much more careful and thorough research and infrastructure.
   b. The idea of asking for risk admin privileges without a) ever submitting a governance proposal b) proving that their models calibrate even in the slightest way to historical events and c) the lack of understanding of how to model liquidations suggests that the OP is flippant with regards to community norms (again as @bgdlabs pointed out in the first post)
   c. Chaos has never submitted a governance proposal to the protocol

3. If the goal is to simply to make developer tools as opposed to providing risk recommendations for billions of dollars of assets, why is the DAO paying for this? Shouldn’t it be funded by developers purchasing these tools (such as BGD or Aave Companies)?

   a. There also doesn’t appear to be any open source code outside of some basic strategies (and as I outlined above, it seems extremely difficult to trust that Chaos’s understanding of building such models is beyond mediocre).
   b. Moreover, what happened regarding Chaos’s engagement with Maker? Are there any results from there that can be used to inform the community of competency (or lack thereof)?

It would, of course, be absolutely amazing for there to be free, open, easy to use software for one to measure risk on their own and be able to validate that the conclusions are correct. And one day, there will be such tools (especially with zero knowledge proofs and FHE), where model developers can give you succinct cryptographically verifiable proofs that
a) their model was trained as stated
b) their generalization error is what they claim
c) the model is run correctly on new data.
Gauntlet definitely spends a lot of time researching and working with zero knowledge researchers (having written two papers on ZK DeFi and cohosting the ZK Podcast) to try to make this a reality and when it is technically feasible, we are excited to provide it to the community.

However, we are extremely far from that point and currently, the user relies on providing their own input and their own models, which can be incomplete or worse, anti-correlated to the correct answer. The scenarios for agent-based modeling are very similar to those for formal verification — I can give you the bazooka to utilize, but even if you have the most powerful weapon, if you can’t operate it correctly, you can easily shoot yourself in the foot. As I stated above regarding the modeling woes of Chaos’s case study and their clear misunderstanding of how to jointly optimize parameters for a complex protocol, it seems rather difficult for any fiscally responsible tokenholder to believe this modeling alone should be used to manage billions in AUM. It would be great to have multiple risk managers — but Aave should optimize for competence and a track record in DAOs, which this proposal does not do (unlike @Llamaxyz’s revised proposal, for example).

[fig]

Well, this is juicy - lots of new accounts created in the past day, and eyes are on the forum because of this discourse. It is candid and passionate which I believe leads to more engaged forum participants.

A few things I would like to call out (as a somewhat outsider): one factor which is over-prioritized in Aave and other DAOs is a “history of engagement.” While this is valuable - it is very rare.

It is hard to find teams that have engaged as long as organizations like Llama or Gauntlet, especially in a nascent industry. As we mature as an industry, we better hope new tools and solutions are being built.

This mindset of automatically discounting new solutions is predatory, deterring innovation, risk-taking, and invested contributors; if we want to see DAOs succeed, we must be open-minded.

(Side note: it’s also a limitation of Governance)

Now on the merit of Chaos - I can not speak to the technical capabilities of the product but it seems to offer the DAO scalability at fewer resources. Additionally, the modified fee structure feels appropriate.

$500,000 after 6 months of work.

If we are looking to grow Aave 10x, it comes with more scalable ways for people to engage. Chaos feels like it offered hints of that, allowing new participants to use their product and do risk analysis.

@tarun

It would be great to have multiple risk managers — but Aave should optimize for competence and a track record in DAOs, which this proposal does not do (

@tarun I would ask how would you recommend new teams and products build this track record? Focus on other smaller DAOs? Work for free and pursue retroactive funding?

Curious about your thoughts here.

Also - if any outside team similar to @DiamondRock has interfaced with both products, please chime in. Seems to be an early instance of these teams overlapping, hence the passionate debate.

[inkyamze]

To address a few of your points:

• Great to see your updated proposal on working for free for six months. We also took on shorter engagements when we started contributing to Aave and found it to be invaluable in establishing a track record with the community.
• There is no vendor lock-in with our Aave engagement as Aave can terminate the payment stream at anytime.
• We welcome alternate viewpoints on risk. As a recent example, we partnered with Block Analitica on risks they identified related to the merge.

More broadly, the biggest issue with your updated proposal is having two distinct risk managers. This doesn’t work for Aave for several key reasons:

There can only be one set of risk parameter recommendations in production

Setting parameters is a joint optimization problem. For example, increasing LTV without an appropriate change in Liquidation Bonus may severely increase the risk of cascading liquidations. Further, increasing LTV on one asset has effects on multiple other assets in Aave given that it is a many-to-many system. If there are multiple risk managers submitting recommendations, Aave would need to be able to aggregate risk recommendations. And in order to this, there needs to be some cohesion over their submissions.

Multiple risk managers creates more risk for the protocol

If Chaos makes parameter recommendations that cause insolvencies, Aave and its users will lose funds.

Which risk manager is ultimately accountable for protecting the protocol? This becomes far less clear with multiple risk managers.

Increased governance overhead and complexity

It’s incredibly valuable to have community discussion on risk-related topics (e.g. pausing ETH borrowing, FEI recommendations). Having two distinct risk managers poses some key questions for the community. How does the community handle situations where one risk manager says risk off, while the other says risk on? How long will a decision like this take to get through governance? What are the downsides for users due to inaction?

---

We want to see more risk management for Aave, but two parties submitting recommendations on the same parameters using two different methodologies and strategies doesn’t work.

If Chaos were to narrow its proposal scope further and remove risk parameter recommendations, then there is no reason why they shouldn’t develop tools for the benefit of the broader community. In this case, Gauntlet will fully support and vote Yes, and encourage others to do so as well.

[omergoldberg]

Snapshot is live here, but we wanted to touch on a few last items.

Multiple risk vendors

• This has been discussed at length throughout this proposal and it is our understanding that the community is supportive of this.
• If the DAO hopes to grow 10x in the near term (as @fig alluded to) we will need to onboard additional vendors across multiple service areas, especially risk. Divergence in parameter recommendations is important and exactly the moments in which additional discussion is critical. Having more voices, more models, and more opinions only increase the robustness of the DAO’s decision-making process.
• This is an opportunity to make results and recommendations more transparent and allow the community to voice an opinion on the balance between capital efficiency and economic security. We hope that this fosters a plan to revisit the RiskDAO concept proposed last year (to which it seemed all relevant parties were supportive).
• Concluding that introducing additional full-time risk contributors for Aave results in higher risk seems counterintuitive to us, the responses in the forums, and the opinions from the majority of the community members with which we have spoken. This obviously isn’t the case for traditional finance, smart contract auditing, web2 cyber security, etc. where redundancies and differences in opinion are key for coverage.
• Chaos is happy to prioritize Aave’s success and collaborate with any risk and security contributor to advance this cause.

Removing Risk Parameter Recommendations from the Chaos Labs Proposal

• The Gauntlet team has made it very clear on this thread that they view Chaos as competitive in parameter recommendations. Chaos views competition as a healthy forcing function for building better, more transparent products at a more competitive price. The real winner of the said competition is the Aave DAO.
• Predicating votes based on whether we remove the scope that an incumbent view as competitive is a clear embodiment of conflicting interests. Furthermore, this is a perfect example of the risks of early vendor lock-in, as stated above.
• This is a partnership that has zero upfront cost to the DAO and allows them to examine alternative solutions freely while shifting the vast majority of the risk to Chaos.
• The spirit behind this proposal was to make this an easy decision for Aave token holders to onboard new DAO contributors as we move forward to further decentralizing protocol risk management and scale Aave as a whole.

To reiterate, Chaos has acknowledged the obvious conflict of interests and will abstain from the vote. We will also ask our investors to do the same so that the community can make a decision about what it believes is best for the protocol.

Our team at Chaos will do all of the work upfront and showcase its capabilities. Once that has been delivered and battle-tested, the DAO can decide to retain its services at a predetermined price or terminate before any payment is made. This gives certainty to Chaos and flexibility to the DAO.

We’re looking forward to seeing the community’s feedback and engaging further!

[eboado]

First of all, given that there is some “intense” discussion on the thread, I would like to clarify that I have no specific relation (e.g. no investment or anything of the sort) with Gauntlet or Chaos Labs. So my opinion only represents what I think is good for the Aave DAO.
In addition, I was against the initial Chaos Labs proposal, but I don’t really have a decision taken at the moment about the updated one, even if I will explain on the following post some negatives points imo.

@omergoldberg

Multiple risk vendors

Let’s try to be a bit serious and stop with this type of argument, thinking that a community, because of being decentralized and having important coordination matters to improve, is so naive.
Precisely because of the aforementioned, having 2 (or more) different parties, with 2 different systems to do for example recommendations, will just lead to 2 potential results:

1. Continuous conflict of interest on what is better for the protocol.
2. One of the parties just not doing much.

What Aave does needs, apart obviously from multiple contributors, is optimality. It is too easy to say “being a big protocol, you need as many risk vendors as possible”, but 1) not true, what Aave needs is a good vendor, compromised, showing continuous results (and additional independent contributors, via grants for example) 2) the Aave DAO is the main “victim” of these “we are all friends, anyway the Aave DAO pays”.

---

This is pretty simple, when proposing a continuous engagement with the DAO, not even presenting as a conflict of interest existing collaborations of the DAO (until other people comments), should be understood as disrespectful. It creates an important cognitive overhead for all discussion participants, as they are not fully informed of all details around, them and obviously not risk/technical experts.

@omergoldberg

The spirit behind this proposal was to make this an easy decision for Aave token holders to onboard new DAO contributors as we move forward to further decentralizing protocol risk management and scale Aave as a whole.

That leads to members of the community being forced to address the “elephants in the room”, because the OP doesn’t.

---

@omergoldberg

Our team at Chaos will do all of the work upfront and showcase its capabilities. Once that has been delivered and battle-tested, the DAO can decide to retain its services at a predetermined price or terminate before any payment is made.

I still don’t really understand this. There is something called Aave Grants DAO, giving millions of dollars to participants in the ecosystem. Let’s try to not normalize or adopt any “victimized” position on what is not: the issue here is not (only) about the budget, is about the fundamentals. It is almost certain given the previous grantees that the contributions of Chaos Labs would be really strong candidates to receive grants. From what I know, it was already the case.
So this argument of “we found big opposition, so we change the model of $3m to free 6 months (and still a pricing model afterwards)” is just not strong, and only helps to set a “soft obligation” on the DAO to engage after the 6 months.

---

Let’s be clear again, I’m probably one of the main critics in the past of Gauntlet (and support when good work was done ofc), and honestly, the name behind the risk vendor doesn’t really matter to me, if it is fully professional and optimal in all senses for the DAO. In addition, Gauntlet didn’t try to “lobby” me, apart from professional collaboration with BGD (and everybody can be sure that lobbying would matter more or less 0 to me, my interests are with the DAO).

But since reading the initial proposal, my clear impression is:

• No proper due diligence of procedures on how everything is at the moment on the Aave DAO (could be good to answer on who is the non-existent AAVE risk team Chaos Labs - Risk & Simulation Platform Proposal - #16 by PennBlockchain).
• Quality project on top of Aave, but really early. Basically same as dozens going through the Aave Grants DAO.
• Speculation on fields to cover (GHO, community contribution, etc), without really understanding what the community requires at the moment.

If Chaos Labs really wants to do parameters recommendation, it should create a Snapshot for the Aave community to choose if parameters recommendation should be done by Gauntlet, Chaos Labs, or whoever else shows will/credentials.

[PennBlockchain]

Love to see all the discussion in the forums around this proposal and as @fig pointed out, the creation of new accounts around any post goes to show the importance of such a topic.

For full transparency, we’ve been in pretty close contact with both the Chaos team and Gauntlet who have raised some interesting and serious concerns the past few weeks.

After honestly weeks of edits and discussions, we have come to our conclusion, and that is an ultimate YES on the current snapshot vote. We believe the past work that Chaos Labs has done is very good and goes to show the level of due diligence we should not only expect, but hold the team to.
From the first draft of the proposal to now, a lot has changed. Ideally, we would have loved to see more GHO analytics, but seeing concerns about the uncertainty in timeline, the removal of that makes total sense. The main thing that brought this over the finish line for us was the 6 months of no cost, and then ability to off-board with no to minimal costs if they provide unsatisfactory work.

To conclude, the work Chaos Labs has done, the promises they make for this engagement period (6 months of which are technically free), and “sneak-peaks” into their plans lead us ultimately to vote YES. Everything else has been pretty much covered in the forums, but we always enjoy an “underdog” story and are generally open to ambitious teams trying to make a name for themselves given the circumstances!

[rspa]

Hello Aave community, Raphael from Flipside Governance here, this is my first post.

I’ve had the pleasure to be on a call with @omergoldberg when ChaosLabs applied for a core unit position at MakerDAO, where we are delegates.

While we felt that a CU position was not the right path for Chaos Labs to pursue, and would have preferred them selling their services to the respective existing core units instead, I want to say this:
I was deeply impressed by the quality of thinking I encountered and I’m confident that Chaos Labs can deliver what they promise.

See the testimony from BenQi in this thread

@DiamondRock

we felt important to share our experience, which has been amazingly positive with Chaos Labs. The Chaos team is excellent, honest, and hard-working. In my opinion, they are world class and have guided Benqi in times of high market volatility with top-notch risk tooling and simulations.

Chaos Labs’ offer to prove their merit for free for a half year is a further testament to their conviction.

Instead of engaging in a back-and-forth on the various risk models between Gauntlet and Chaos Labs, both of which are closed source and confidential, I’d propose that the DAO simply tries Chaos for 3-4 months and then reconvenes for an assessment of the merits of their product or lack thereof.

After that time, the community can base their judgment on the reality of having ChaosLabs as a partner in Aave and not on a theoretical discussion.

[omergoldberg]

Thank you for the overwhelming community support!

We are drafting the on-chain proposal and will be posting it soon!

[omergoldberg]

Hey all - the on-chain vote will begin in ~3 hours here.

We appreciate all the support on the Snapshot vote. We’re excited to begin contributing full-time to the Aave community!

We’ve been working on a few things that we’re excited to share over the coming weeks

[OriN]

Roadmap Update

Following the successful governance vote, Chaos Labs is thrilled to kick off our engagement with the Aave community. The team has begun working on all fronts to deliver high-quality tools and recommendations to ensure the protocol’s continued growth and safety. We are grateful for the overwhelming support and look forward to building a state-of-the-art platform that will empower the community to make data-driven decisions.

In this post, we will detail our roadmap for the first months of the engagement.

Roadmap

Our primary focus for the next 8-10 weeks will be on the first pillar of the engagement - Risk Parameter Recommendations for AAVE V3 markets, including LTV, Liquidation Threshold, Liquidation Bonus, and Supply Caps. The recommendations portal will be a community-facing, public dashboard allowing users to better understand the tradeoffs between specific parameters and transparency into the more detailed simulation methodology.

As the community discussion and vote were underway, our team started the model expansion and testing process to fully support Aave V3 and deliver a full suite of risk products to the Aave community.

We are currently iterating and enhancing the existing simulation engine and models:

• Expanding data pipelines and analytics to incorporate key AAVE v3 data points, including (but not limited to) enriched wallet data & profiling, asset correlations & volatility, on-chain, and off-chain liquidity, and trading volume
• Integrating AAVE-specific features and data into our Monte-Carlo simulation framework
  • Conducting research and data analysis to continue to identify the relevant agents and behaviors in the AAVE ecosystem
  • Emulating arbitrageurs’, market makers’, and liquidity providers’ effects on off-chain and on-chain liquidity; Borrowers’ behavior based on historical patterns; Liquidators’ operation across CeFi and DeFi venues under liquidity constraints (price impact and slippage)
• User-friendly and comprehensive Parameter Recommendation Dashboard to display simulation data, results, and recommendations

In addition to the above, we have been working on the Chaos Price Manipulation Tool to help better understand the feasibility and implications of market manipulation attacks (i.e., Mango) on AAVE markets. We view this tool as a critical component in determining protocol risk and will be an effective “white hat” tool to protect the protocol better. In the next few weeks, we will share more regarding functionality and access to help the community align on safeguards to this specific protocol attack surface.

What’s Next?

Once the initial phase of Aave V3 risk tooling is launched, we will continue to enhance the suite of products with the additions of:

• Continuous parameter recommendations while incorporating community feedback and preferences into our models and simulations
  • Expanding the parameter recommendations tool to provide support and analysis for E-Mode, Borrow Caps, and Interest Rate Curve changes
  • Open-sourcing agents and simulation model transparency initiatives for feedback and public iteration so that all Aave stakeholders can contribute to securing the protocol with new agents and modeling techniques
• Asset Listing Portal - building on the parameter recommendations infrastructure, we will deliver tools to help streamline new collateral onboarding to the Aave protocol, which will provide insights into initial parameter and mode classification recommendations while projecting revenue opportunities for proposed assets

Communications

We will provide a monthly update post focusing on complete and ongoing works as determined by the community. Alongside major releases or ongoing risk assessments, we will host regular community calls and office hours for platform feedback and discussion.

We invite community feedback and would love to hear any comments and questions!

We are continuously monitoring the market and Aave pools for potential risks and will communicate any concerns we see proactively in the forums. Additionally, we are working with our relevant risk counterparts to discuss any proposals or updates to the protocol that might impact the security of user funds.

[G-Blockchain]

Personally, I am excited about the introduction of the regular community calls and office hours to see the platform

Is there an ETA for the first one?