Pinned post
Excited to share first glimpses of our research on agent collectives, a year in the making.
I’ve been thinking about parallel reasoning for some time since I was at Meta. Why automate researchers when we should be automating communities? Genius is great, but scenius matters too.
We’re sharing a recipe today (RLCR) that is producing interesting results for us at a small scale. We’re using it to train our Avalon-M1 series of models, starting with a small 120B model.
Usual caveats apply. We are a small team (8 people!), we are conducting more experiments and ablations as we speak, and we’re still training and seeing gains.
Let us know your feedback. My summary of findings in thread below. 🧵
I’ve been thinking about parallel reasoning for some time since I was at Meta. Why automate researchers when we should be automating communities? Genius is great, but scenius matters too.
We’re sharing a recipe today (RLCR) that is producing interesting results for us at a small scale. We’re using it to train our Avalon-M1 series of models, starting with a small 120B model.
Usual caveats apply. We are a small team (8 people!), we are conducting more experiments and ablations as we speak, and we’re still training and seeing gains.
Let us know your feedback. My summary of findings in thread below. 🧵
Introducing Reinforcement Learning from Collective Rewards (RLCR).View quoted post on X
RLCR is a recipe that trains language models as agent organisations, which share a message board and workspace. We use it to train Avalon-M1-Mini (120B) and study its behaviour.
We find a number of interesting results, including:
🛠️ Learnt organisational hierarchies and divisions of labour
🕵️ Increased collective verification, through hires of verifiers and testers
👥 The rise of collective thinking, where agents think more about other agents and how to help them
We believe RLCR can provide a laboratory for studying collective intelligence and multi-agent safety.
Blog post and technical report 👇