@PeterStone_TX
Large repos often have 1000+ devs working at once. What happens when devs delegate to agents?
We introduce ParallACT, a benchmark of multi-principal multi-agent tasks requiring coordination.
Frontier models struggle (e.g. 16% success for Opus 4.8), even when they can talk.
This setting differs fundamentally from most current multi-agent settings, which decompose a single problem and assign to parallel agents. These problems all started as a centralized request. In contrast, ParallACT is multi-agent by necessity: the requests come from different principals, and sometimes happen to conflict. This happens all the time, e.g. with devs working on monorepos.
Key takeaways:
1️⃣ Parallel agents fail on the same tasks where a single agent succeeds: on contradictory specs, single agents given both tasks achieve 62%, vs 16% for two agents working in parallel (46 pp drop). This holds across models and harnesses. Often each parallel agent’s half works on its own, but the combined code doesn't.
2️⃣ Communication helps a little and costs a lot: the best message channel reaches 32%, at more than twice the tokens. Just knowing a peer exists doesn't help.
3️⃣ Agents know what to talk about, but not how to talk: they communicate about the right code but read messages too late, and often still make conflicting edits after agreeing on a design.
4️⃣ Taking turns recovers much of the gap, but doesn't scale beyond a few agents. Sequentially iterating between agents lets them coordinate better but requires central orchestration.
🧵👇View quoted post on X
I recently defended my PhD on how RL agents can learn with and from other agents!View quoted post on X
I’m truly grateful to my advisor, Peter Stone, who has shown me how to conduct research with vision, integrity, and rigor, while fostering a great environment for collaboration and growth. I’m also grateful to the many amazing people I’ve encountered along the way, including my mentors, who believed in my ideas while teaching me how to make them better, my labmates, many of whom became valued collaborators and friends, and my mentees, for their fresh perspectives and energy. I’m also grateful to my partner Ben, for his unwavering support along the way.
If you’d told me at the beginning of my PhD that I would be sad when it was over, I wouldn’t have believed you. But I’m indeed sad to move on from LARG, which has become my home.
I’m excited for the next phase of my life, where I’ll be working on multi-agent systems at Google DeepMind London, as a research scientist!
[1/n] I’m thrilled to share that the ROTATE paper, which casts learning generalizable coordination behaviors as an open-ended learning problem, has been accepted to #NeurIPS26 as a spotlight 🌟! See you in Sydney!
Collaborators: @ArrasyRahman, Benjamin Nativi, Johnny Liu, @cuijiaxun, @yoonchangsung, @PeterStone_TX.
Paper: arxiv.org/abs/2505.23686
Code: github.com/carolinewang01…
For those of you who might have seen earlier versions of the paper, in the most recent version, we made substantial advances by characterizing the self-sabotage problem as per-state incompetence, and viewing ROTATE as placing a competence constraint on learning, therefore improving our understanding of why ROTATE works. Details in the thread below.View quoted post on X
