Twitxy Saved

Kristen Kehrer

@DataMovesHer

Data Scientist & Children's Book Author
Littleton, MA datamovesme.com Joined March 2010
1.4K Following 3.2K Followers 2.9K Posts

Latest posts 5

Kristen Kehrer @DataMovesHer
We're all over here using LLMs to code and feeling like rockstars.. meanwhile, those same models trip over tasks scientists handle routinely.

@joinHandshake tested 10 frontier models on VIALS, a benchmark of 161 interpretation tasks from real lab workflows.

Nothing cleared 27%.
Jonas Mueller Verified account @jomulr Aug 25
AI promises to cure disease, but can frontier models interpret the images biotech/pharma scientists handle every day?

To evaluate this, we built the ATLAS Visual Life Sciences (VIALS) benchmark: 161 visual interpretation tasks from professional life sciences workflows including gel blots, plasmid maps, molecular structures, and phylogenies. These are the images/results that experiments actually produce, not polished figures from papers or textbooks.

We benchmarked today’s 10 leading multimodal models. Opus 5, GPT 5.6, and Gemini 3.7 all fail to crack 27% accuracy, and the other models like Kimi K3 & Grok 4.6 perform much worse.

In contrast, PhD-level biomedical scientists find these visual interpretation tasks straightforward. AI that cant interpret such images will have limited utility in life sciences R&D, where such artifacts are central to how scientists reason, communicate, and make research decisions.

We hope VIALS drives progress toward models that can.
View quoted post on X