Community Newsletter 2
The following is an archive of the ARC Prize Community email newsletter. Subscribe to future emails.
Win hackathons with ARC-AGI
Hello, AGI enthusiasts!
ARC Prize is picking up steam. Your posts, discussions, and work on the only benchmark that measures our progress towards AGI are bringing fresh momentum to the field of AI.
Let’s jump right into it.
Activity
Congrats to Aditya Advani and team for winning 3rd place at the AGI House in San Francisco! Mike Knoop spoke at the event with other AI influencers including George Hotz of Comma AI/tinygrad.
Reach out to us if you're looking to build an event around ARC-AGI.
Besides astounding engagement and lots of smart ideas, the community has also grown significantly in numbers over the last two weeks.
- YouTube: 20,532 views (+56%)
- Discord: 1,422 members (+24%)
- Twitter/X: 3,051 followers (+26%)
- Kaggle: 6,539 entrants (+48%), 434 teams (+237%), 1,882 submissions (+266%)
Content
Training Data Podcast by Sequoia Capital
Mike Knoop was on the Training Data podcast by top-tier venture firm Sequoia Capital, diving deep into the fundamental ideas behind ARC-AGI, promising approaches, and the future impact of AGI.
Listen: YouTube, Spotify, Apple
You can also find the follow-up essay here: The New Ideas Needed for AGI.
Sean Carroll's Mindscape Podcast
François Chollet joined Sean Carroll on the Mindscape podcast to discuss deep learning and the meaning of intelligence, including LLMs and whether or not they will lead to general intelligence.
Listen: YouTube, Spotify, Apple
Community Content
We’re excited to see the AI community engaging with ARC Prize and creating valuable videos and discussions. Here are a few highlights.
- AI Won't Be AGI, Until It Can At Least Do This (plus 6 key ways LLMs are being upgraded) - AI Explained on what LLMs today miss in the context of ARC-AGI.
- Chollet's ARC Challenge + Current Winners - a Machine Learning Street Talk interview with ARC-AGI world record holders Jack Cole, Mohammed Osman, and Michael Hodel.
- Claude 3.5 struggle too?! The $Million Dollar Challenge - AI Jason discusses different methods to possibly solve ARC-AGI.
Updates
In case you missed it, we have reduced the daily submission limit from 5 to 3 to prevent overfitting and maintain competition integrity. This change encourages thoughtful iterations and aligns with best practices in machine learning competitions.
Read the full announcement: Daily Submission Limit Change
ARC-AGI-Pub is Live
If you want to test model performance on ARC-AGI no limitations on internet access and few limitations on compute, you can submit your solution to the ARC-AGI-Pub leaderboard.
We are committing $150,000 USD toward a verification fund for this new leaderboard to help reimburse resource costs.
Note: ARC-AGI-Pub is not part of ARC Prize 2024 and has no associated prizes.
Responding to numerous questions on Discord and Twitter - the ARC-AGI-Pub score is measured on the public evaluation set, while the semi-private evaluation set used to check against hardcoding solutions to the public tasks is a new and (for now) hidden dataset.
Read the full announcement: Introducing the ARC-AGI Public Leaderboard
Team Up
As promised, we feature participants who are looking for teammates on the ARC Prize Discord in our bi-weekly newsletter. Here are some members seeking collaborators.
- @Andreyz4k is looking for teammates to help improve a DreamCoder-based solution for neurally-guided discrete program search, needing assistance with performance optimizations, introspection tools, expanding the task dataset, and improving the neural network and Stitch subsystem.
- @nilskakoseos, a solo mathematician, has developed an early-stage, high-level solution plan for ARC and is seeking collaborators with interesting views or approaches, open to working on any part of the solution or joining other teams.
- [ARC-AGI-Pub] @manny is looking for collaborators to build a solution tailored for Claude Sonnet 3.5, based on the current top-scoring solution on the leaderboard, believing that improvements can be made over the handcrafted solution for GPT-4o, which struggles with instructions, prompt adherence, image recognition, and context length.
Jump into the #team-formation channel in Discord to find your team.
Task of the Week
Follow us on Twitter/X and add an interesting ARC puzzle to your daily Wordle routine. 😎 Are you still smarter than AI? Find out.
We look forward to many more discussions, submissions, and world records in the coming months. Will you be the one to solve ARC-AGI?!
See you around!
Oliver
ARC Prize Team
P.S. Creativity to pave the way! "Instructions unclear" by @hysminea on Discord.
Be the first to get the latest ARC Prize info. Subscribe now.