Twitxy Saved

Lukas Bergstrom

@lukasb

AI Evals & Benchmarks @ Zapier
San Francisco, CA lukasbergstrom.com Joined January 2007
1.8K Following 1.6K Followers 3.5K Posts

Latest posts 5

Lukas Bergstrom @lukasb
Great to see! Having another SOTA lineage benefits everyone. Have been looking forward to showing the new AutomationBench leaderboard with Gemini in the top 5 spots: zapier.com/benchmarks
zapier.com AutomationBench: AI Agent Benchmarks | Zapier Zapier's AI benchmark assessment scores LLMs on end-to-end workflow execution across real tools and business areas. Deterministic grading. No vibes.
Logan Kilpatrick Verified account @OfficialLoganK 1d
Introducing Gemini 4 Neon! Pushing the frontier across domains and punching above its model class in a meaningful way. To me, Neon feels like the right balance of intelligence, speed, and usability. Available now across @GoogleAIStudio, the API, and more!
View quoted post on X
Lukas Bergstrom @lukasb
Had fun! Even though we chewed through my prepared questions in the first twenty minutes 😅

One thing I realized today is that innovation and model improvement isn't something that just comes out of the labs, it's a collaboration across labs, people building benchmarks, users...
Kaggle @kaggle Oct 28
@zapier, one of our Bench Summit partners, led a panel discussion on the realities of managing highly influential evaluation signals.

The panel, moderated by Lukas Bergstrom (@lukasb), included insights from Robin Salimans (@SalimansRobin) of Zapier, Greg Kamradt (@GregKamradt) of ARC Prize Foundation, and Shaiyon Hariri of Ramp.

Attendees heard about what defines a successful benchmark, the challenges of maintaining adoption, and critical lessons learned from V1 to V2+.
View quoted post on X