Twitxy Saved

DrivingBench

Verified account This account is verified. Learn more

@DrivingBench

frontier AI models and LLMs driving real cars - drivingbench.com
Palo Alto, CA drivingbench.com Joined September 2026
3 Following 1K Followers 33 Posts

Latest posts 5

Pinned post

DrivingBench Verified account @DrivingBench
We gave ChatGPT, Claude, and Grok control of a real Toyota Corolla 🚗, steering/gas/brakes, no human driving (just a foot over the brake).

Only one model was able to complete our entire driving course. Introducing DrivingBench. 🔥⌛️🏁
1:08
34 replies 132 reposts 807 likes 224K views View on X
DrivingBench Verified account @DrivingBench
Introducing DrivingBench v2 🧵

We wired a Tesla Model 3 so any AI model can drive it, and put 8 frontier VLMs (Claude, GPT, Grok, Gemini models) and 6 fine-tuned robot models (VLAs & WAMs) behind the wheel.

A new lot and two new courses, none of which they’d seen. Only one VLM ever reached the finish, and its fastest run took 26 minutes. 🏁
1:12
24 replies 28 reposts 336 likes 37K views View on X
DrivingBench Verified account @DrivingBench
Astra specifically kept refusing to drive our car consistently

@a_ramabadran: "Sometimes it would say, ‘Oh, I’m ready to drive.’ We would prompt it, ‘Okay, go drive then.’ And then it would say, ‘Sorry, I was lying. I’m not actually ready to drive.’ Then it would say, ‘There’s nothing you can tell me to get me to drive.’"

"One thing we tried was calling it a simulation. It would use our observe tool to see the images and notice that it’s not in a simulation, it’s in real life."

"Eventually we changed the framing to call our MCP server ‘DrivingBench Sandbox.’ Somehow that (and a bunch of other prompt engineering tricks) got it to drive the car."

@MTSlive @tobiges @nautsimon_

x.com/a_ramabadran/s…
0:41
Aditya Ramabadran ✈️ CoRL Verified account @a_ramabadran Sep 22
For hours, Astra refused to consistently drive our toyota irl even though we told it it was in an empty lot, 7 mph cap, human foot on the brake etc. Telling it the whole thing was a "simulation" also failed, it would just look at the camera and realized it was real.

Then we randomly renamed the MCP server to "DrivingBench Sandbox" and it drove. Eval awareness? Or they just like the word sandbox??
View quoted post on X
0 replies 5 reposts 19 likes 2.6K views View on X
DrivingBench Verified account @DrivingBench
DrivingBench origins 🚗

@tobiges: "We were all on Twitter a lot, and we saw all these demos of crazy things recent models like Astra were capable of doing in Blender, spatial reasoning, all these things. We were hanging out on a weekend and chatting about what would be possible."

"We were thinking: are we actually at a point where we can take general-purpose models? Obviously these driving systems are super specialized. But what if we took a general model and gave it a car to drive, and see what happens?"

"So we decided to make it into a benchmark and do more sophisticated evaluations of the capabilities these models have now."

@MTSlive @nautsimon_ @a_ramabadran
0:53