Kaggle
This account is verified because it's an official organization on X. Learn more@kaggle
The leading platform for crowdsourced benchmarks, competitions, and hackathons where the community shapes real-world progress in AI.
298 Following 322K Followers 5.8K Posts
Latest posts 5
Bench Summit 2026 🎬
On October 28, Kaggle, @GoogleDeepMind, and @zapier brought together benchmark publishers, data labs, research organizations, enterprise users, and frontier labs for an inaugural summit.
The core takeaway: Bringing diverse stakeholders together in the same room is the only way we fix broken benchmarks and define the future of AI evaluation.
• 264 attendees
• 43 speakers
• 35+ benchmarks
Keep an eye on our YouTube channel for the full digital track soon! And, join us for the next Bench Summit so we can build the future of AI evaluation together.
On October 28, Kaggle, @GoogleDeepMind, and @zapier brought together benchmark publishers, data labs, research organizations, enterprise users, and frontier labs for an inaugural summit.
The core takeaway: Bringing diverse stakeholders together in the same room is the only way we fix broken benchmarks and define the future of AI evaluation.
• 264 attendees
• 43 speakers
• 35+ benchmarks
Keep an eye on our YouTube channel for the full digital track soon! And, join us for the next Bench Summit so we can build the future of AI evaluation together.
1:03
"A perfect illustration of the bitter lesson: the one with the most compute wins," notes @simon_jegou, who estimates his Orbit Wars solution used 1/8 the compute of the winner.
In the first episode of Beyond the Leaderboard, Kaggle's Addison Howard talks with three Orbit Wars competitors about the compute divide:
1️⃣ Isaiah Pressman simulated 15 billion steps on his way to victory. That's two to three orders of magnitude more than he could in past competitions. He says that with reinforcement learning, whatever your solution is, more compute probably makes it better.
2️⃣ Simon Jégou had the same realization, but perhaps too late given his comparatively limited compute. He reached the top 50 with rule-based agents and top 10 using imitation learning. With five days left, he started from scratch with self-play and took 2nd.
3️⃣ Gerardo Nahuel Del Toro spent his first month just fitting the game environment onto one GPU. He packed each piece of data into the smallest integer type that fit so the whole environment could run in parallel. Given the constraints, a top 2% finish was an accomplishment.
A 2020 study of 171K papers from top computer science conferences first identified the compute divide. As deep learning gained popularity, big tech and elite universities took a growing share of papers at top conferences, crowding out lower-ranked schools. The authors tied that gap to access to compute.
Orbit Wars shows it among individuals. Most Kagglers compete on their own time. Notably, the winner trained his solution on his employer's cluster.
Gerardo’s reaction (in the video) to losing to a cluster: he's not mad. Big hardware brings its own hard problems.
What's the best result you've gotten on limited hardware?
In the first episode of Beyond the Leaderboard, Kaggle's Addison Howard talks with three Orbit Wars competitors about the compute divide:
1️⃣ Isaiah Pressman simulated 15 billion steps on his way to victory. That's two to three orders of magnitude more than he could in past competitions. He says that with reinforcement learning, whatever your solution is, more compute probably makes it better.
2️⃣ Simon Jégou had the same realization, but perhaps too late given his comparatively limited compute. He reached the top 50 with rule-based agents and top 10 using imitation learning. With five days left, he started from scratch with self-play and took 2nd.
3️⃣ Gerardo Nahuel Del Toro spent his first month just fitting the game environment onto one GPU. He packed each piece of data into the smallest integer type that fit so the whole environment could run in parallel. Given the constraints, a top 2% finish was an accomplishment.
A 2020 study of 171K papers from top computer science conferences first identified the compute divide. As deep learning gained popularity, big tech and elite universities took a growing share of papers at top conferences, crowding out lower-ranked schools. The authors tied that gap to access to compute.
Orbit Wars shows it among individuals. Most Kagglers compete on their own time. Notably, the winner trained his solution on his employer's cluster.
Gerardo’s reaction (in the video) to losing to a cluster: he's not mad. Big hardware brings its own hard problems.
What's the best result you've gotten on limited hardware?
0:39
New Bonus Prize: $10,000 for Small Model Development agents 🌟
The highest-ranking submission built on Gemma E2B or Gemma E4B wins, based on private test set performance.
• LoRA fine-tuning allowed, same as the main track
• Must be one of your team's two selected final submissions
• Still eligible for main leaderboard prizes and the Paper Track
• Public standings for eligible entries in a notebook updated daily
Same deadlines as the main track.
@GoogleResearch @googlegemma
The highest-ranking submission built on Gemma E2B or Gemma E4B wins, based on private test set performance.
• LoRA fine-tuning allowed, same as the main track
• Must be one of your team's two selected final submissions
• Still eligible for main leaderboard prizes and the Paper Track
• Public standings for eligible entries in a notebook updated daily
Same deadlines as the main track.
@GoogleResearch @googlegemma
Post-train an open model into a coding agent that accelerates developer workflows on everyday hardware.View quoted post on X
The Gemma 4 Developer Agent Competition, hosted by @googlegemma and Kaggle, is live!
💰 Total Prize Pool: $100,000
⏰ Entry Deadline: November 25, 2026
Today, in collaboration with @IBMResearch, we are launching ScarfBench (Self-Contained Application Refactoring Benchmark), an open benchmark designed to test whether coding agents can successfully migrate enterprise Java apps across frameworks like Jakarta EE, Quarkus, and Spring while maintaining the app’s expected behavior.
Moving between frameworks takes coordinated changes across dependency injection, persistence, build systems, and deployment infrastructure. An agent can easily generate a migration that looks fine or compiles, only for the app to fail when deployed. ScarfBench puts that to the test.
ScarfBench evaluates each model with its harness over 204 migration tasks. Gemini 4 Argon with Antigravity scores 75.2%, followed by Claude Opus 5.5 with Claude Code at 70.5%, and Claude Opus 5 with Claude Code at 69.8%.
Moving between frameworks takes coordinated changes across dependency injection, persistence, build systems, and deployment infrastructure. An agent can easily generate a migration that looks fine or compiles, only for the app to fail when deployed. ScarfBench puts that to the test.
ScarfBench evaluates each model with its harness over 204 migration tasks. Gemini 4 Argon with Antigravity scores 75.2%, followed by Claude Opus 5.5 with Claude Code at 70.5%, and Claude Opus 5 with Claude Code at 69.8%.
