François Chollet Launches Nonprofit to Pioneer AGI Benchmark Standards
Renowned AI researcher and former Google engineer François Chollet has co-founded a groundbreaking nonprofit organization dedicated to developing benchmarks for measuring artificial general intelligence (AGI). The ARC Prize Foundation aims to create standardized evaluations that assess whether AI systems can achieve human-level cognitive abilities.
The Vision Behind ARC Prize Foundation
The nonprofit will be led by Greg Kamradt, former Salesforce engineering director and founder of AI product studio Leverage, who will serve as president and board member. Fundraising efforts are set to begin later this January.
In a statement on the organization’s website, Chollet explained: “We’re evolving into a proper nonprofit foundation to serve as a guiding light toward artificial general intelligence. Our mission is to inspire progress by addressing the gap in basic human capabilities that current AI systems lack.”
Building on the ARC-AGI Benchmark
The foundation will expand upon Chollet’s existing ARC-AGI test, which evaluates an AI system’s ability to:
- Acquire new skills beyond its training data
- Solve novel, puzzle-like problems
- Adapt to unfamiliar challenges through abstract reasoning
First introduced in 2019, the Abstract and Reasoning Corpus for Artificial General Intelligence presents unique challenges that even advanced AI systems struggle with. While current models can excel at specialized tasks like Math Olympiad problems, the best-performing AI to date has only solved about 30% of ARC-AGI tasks.
Addressing Current Limitations and Future Goals
Chollet acknowledges the benchmark’s imperfections, noting that some models have achieved high scores through brute-force computation rather than genuine intelligence. He emphasizes:
“Unlike most AI benchmarks focused on superhuman performance, we’re measuring the gap between AI and human capabilities. Future versions of ARC-AGI will work to close this gap entirely.”
Recent developments include:
- A 2024 competition co-launched with Zapier’s Mike Knoop
- OpenAI’s unreleased o3 model achieving a qualifying score (albeit with massive computing power)
- Plans for a second-generation benchmark in Q1 2025
The Evolving AGI Landscape
The foundation enters a field where even the definition of AGI remains contested. Some industry voices, including an OpenAI staff member, have claimed AGI already exists if defined as “AI better than most humans at most tasks.” Meanwhile, OpenAI CEO Sam Altman has expressed interest in collaborating with the ARC-AGI team on future benchmarks.
The ARC Prize Foundation has outlined ambitious plans to:
- Establish an academic network for AGI research
- Form partnerships with leading AI labs
- Develop industry-wide benchmark standards
- Create third-generation evaluation tools
As the debate about AGI measurement continues, Chollet’s nonprofit aims to bring scientific rigor to one of AI’s most fundamental questions: how will we recognize true artificial general intelligence when it arrives?
📚 Featured Products & Recommendations
Discover our carefully selected products that complement this article’s topics:
🛍️ Featured Product 1: Astonic – Sofa – Blue Leather
Image: Premium product showcase
Professional-grade astonic – sofa – blue leather combining innovation, quality, and user-friendly design.
Key Features:
- Professional-grade quality standards
- Easy setup and intuitive use
- Durable construction for long-term value
- Excellent customer support included
🔗 View Product Details & Purchase
🛍️ Featured Product 2: Arwen – Square Modular Pillow Back Lounge Chair – White Beige
Image: Premium product showcase
Premium quality arwen – square modular pillow back lounge chair – white beige designed for professional use with excellent performance and reliability.
Key Features:
- Cutting-edge technology integration
- Streamlined workflow optimization
- Heavy-duty construction for reliability
- Expert technical support available
🔗 View Product Details & Purchase
💡 Need Help Choosing? Contact our expert team for personalized product recommendations!
