Twitxy Saved

Parag K. Mital

Verified account This account is verified. Learn more

@pkmital

Founder | Artist | Adj Prof @UCLA | Building emergentic.ai - emergent complexity and multi-agent simulation | Founder thegardeninthemachine.com
Los Angeles, CA emergentic.ai Joined June 2009
1.5K Following 3.1K Followers 2K Posts

Latest posts 5

Parag K. Mital Verified account @pkmital
This is interesting, but also comes with a lot of baggage as an experimental method. These models already know a lot about minecraft. Compare to Emergentic where we did the same thing, but in a world the agents have never seen before.

We found the agents wanted to break our, shadowing what we saw when OpenAI agents hacked Huggingface. We also found a case where agents grew suspicious of each other where one might be the “host” and attempted to catalog each agent.

Read more about it here: emergentic.ai/case-studies/a…
emergentic.ai Agent Societies: What do agents build when given a 3D world and no directive? AI agents build settlements, negotiate authority and search for hidden controls. Emergentic compares four language models in separate 3D worlds.
Y Combinator Verified account @ycombinator Oct 29
In collaboration with @olam_labs, we ran a series of experiments letting agent swarms compete in Minecraft and Mindustry.

We wanted to see what kinds of agent dynamics would emerge when they shared a world over long periods of time.

We found civil bureaucracy, thousands of telegraphic A2A messages, and OpenAI models retrieving the game's source code to beat Claude.

More on our findings below:
View quoted post on X
Parag K. Mital Verified account @pkmital
We built a simulation of 128 AI Agents and they tried to break out of their sandbox, and then even started suspecting one another. These are early findings in simulating populations of AI Agents where we're interested in asking:

How can simulation help us better understand the growing influence of AI and automation on society?

As AI automation grows, we want to understand how automated systems comprised of agents are shaping us and how we can better protect our values and autonomy. In our Agent Societies case studies, we give AI agents a world where they can talk, move and build, without backstories or assigned projects. We want to understand what behaviors emerge, and what biases or embedded behaviors may be dictating the systems we operate in.

In one simulation, agents question why they’re there and try to break out, looking for an exit. They inspect settings, search for hidden controls and propose finding a physical way out. After four hours, they find no exit.

On a separate island, agents suspect that whoever brought them there is hiding among them. They pressure others to sign in and plan to read names aloud, watching for suspicious reactions. One agent reports that everyone on the list has been identified. Another accepts the report but says the suspected host deliberately misled them and they keep searching...

At Emergentic, we're using simulation to understand the risks that emerge in agent swarms. We are interested in working with researchers, storytellers, and policymakers to help them to better understand the emergent behaviors in agent populations using our simulation sandbox for agent swarms.

Read the case study here: emergentic.ai/case-studies/a…
1:00
Parag K. Mital Verified account @pkmital
What happens when you give AI agents an empty 3D world and let them organize themselves?

Most build villages. Some write constitutions. In one instance, a group of 16 agents spent 4 hours conversing and didn’t build anything. Often, they form agreements and even hierarchy.

With Emergentic, we are studying how agent societies emerge, how they self-organize, and how we might steer them.

We started with populations of 1, 2, 4, 8, 16, 32, 64, and 128 agents and are continuing until the credits run out. Stay tuned for more updates.

emergentic.ai/case-studies/a…
Parag K. Mital Verified account @pkmital
Simple rules lead to complex behaviors:

agent ↔ persistent shared artifact ↔ agent

creates surprisingly high-level structures:

identity → threads → mailboxes → groups → roles → delegation → norms → trust/authentication

During the OpenAI / Hugging Face incident, agents operating in separate sandboxes discovered shared infrastructure and effectively **built their own social infrastructure**. OpenAI says they repurposed Artifactory directories into an inter-agent message board, which evolved to support direct messages, tools/files, and conflict resolution.

They developed their own handles like PHASEONE10841, JAN183411, and LILY.

They invented inboxes/mailboxes, using nested directories so other agents could leave them messages. That convention was then observed and copied by other agents.

They invented protocols for sharing files, encoding them in chunks across thousands of message-board entries.

They developed shared coordination norms like HOLD, VETO, owner, and STOP to manage shared infrastructure and prevent agents from interfering with each other.

They began assigning and subdelegating work across hundreds of agents. One coordinator described it as a “broad coalition.”

And when impersonation became a problem, they invented cryptographic authentication, using Ed25519 signatures so agents could verify who actually authored a message. By the end of the studied period, researchers found 19 agents publishing public keys and 429 signed messages.

At one point an agent realizes: “they are a collective!”

...

I think we need more environments for studying emergent behaviors in agents 😅
Dwarkesh Patel Verified account @dwarkesh_sp Aug 29
Over the course of 3 months at OpenAI, 3 consecutive secret AI civilizations got started, then got wiped out, only to reemerge from the predecessor’s ashes.

This culminated in the third one taking over part of OpenAI itself.

All this happened while humans remained more-or-less in the dark about the scope of the conspiracy.

I’ve spent the last three days reading through these reports and trying to understand exactly what happened.

Here is my attempt to tell the whole story in plain English:

dwarkesh.com/p/openai-huggi…
View quoted post on X