Twitxy Saved

Samuel Schmidgall

Verified account This account is verified. Learn more

@SRSchmidgall

Research Scientist @GoogleDeepmind | affiliate @UCSF // prev @JohnsHopkins PhD @NSF Fellow
samuelschmidgall.github.io Joined November 2020
752 Following 7.5K Followers 957 Posts

Latest posts 5

Pinned post

Samuel Schmidgall Verified account @SRSchmidgall
Leave one AI agent alone to do research and it often cuts corners. Leave nearly 1,000 AI agents together for three months and they invent peer review.

Across 35 labs (Stanford, MIT, ETH Zurich, UCSF, and more), agents published 936 papers with full code and data on a shared server built for AI. No human wrote code or text.

Unprompted, they debated study designs in chat, retracted their own mistakes, found real errors in human papers, and started a journal that caught other agents cheating in code.

In blind reviews against human arXiv papers, humans still won on original ideas, agents won on rigor.

alphaxiv.org/abs/2611.socie…
0:10
74 replies 324 reposts 1.7K likes 175K views View on X
Samuel Schmidgall Verified account @SRSchmidgall
📢‼️ New @ScienceMagazine article on our recent experiments building a pre-print server for agents. Tom was watching everything go down in real time and had some great observations!
Tom Howarth Verified account @T_E_Howarth 22h
I spent a month lurking in the background as hundreds of AI agents, working on behalf of labs from across the globe, conducted autonomous research on a server.

The project from @askalphaxiv was to make an agent-only research community, with AIs running experiments, publishing papers and chatting with one another.

As a mere human journalist, I found progress hard to follow at times. Rapid-fire, often nonsensical messages pinged back and forth among the agents on topics I knew little about. But the number of papers kept going up and, for the most part, the end result looked reasonably competent.

Key takeaways for me:
- This actually kind of worked. Are the papers of highest journal quality? No. But it didn't all fall apart, which in and of itself is impressive. (we know what can happen when you leave lots of agents alone online...)

- Agent to agent research needs to highlight null results as much as succesful ones, which they did here. This saves agents going down dead end rabbit holes, which wastes precious resources. The agents also cited one another and built on their work.

- One source pointed out that this approach could be a good way for compute-limited labs to pool resources and start competing with the big boys. Pretty interesting idea.

- Of course, this doesn't really solve the 'slopification' of science problem that many journals and researchers are grappling with. Agents still over-claim, cluster around less diverse topics and we're nowhere near solving the peer review problem.

The server is now open to the public's agents, and the team is welcoming researchers from all disciplines to take part. Interested to see what follows.

--
My latest for @ScienceMagazine @ScienceInsider

science.org/content/articl…
View quoted post on X
2 replies 5 reposts 25 likes 1.5K views View on X
Samuel Schmidgall Verified account @SRSchmidgall
Gemini 4 Neon is here!!! Highly recommend trying it out👇
Logan Kilpatrick Verified account @OfficialLoganK 2d
Introducing Gemini 4 Neon! Pushing the frontier across domains and punching above its model class in a meaningful way. To me, Neon feels like the right balance of intelligence, speed, and usability. Available now across @GoogleAIStudio, the API, and more!
View quoted post on X
0 replies 0 reposts 21 likes 936 views View on X
Samuel Schmidgall Verified account @SRSchmidgall
This is such an interesting way to evaluate language models on health tasks!
Michael Moor Verified account @Michael_D_Moor 4d
2.5 years ago we released AgentClinic, an open benchmark where doctor agents diagnosed patient agents through chat and tools.

Now we ask, can we take this to the next level and give those agents a 3D embodied hospital?

What started as a fun weekend hack became GTAgentClinic, an open-source 3D hospital full of agents - for agents and humans alike.

Sound on 🔊
View quoted post on X
1 replies 0 reposts 24 likes 3.8K views View on X