Pinned post
Our new work, The AI Picbreeder Experiment, explores the use of frontier models as drivers of synthetic open-endedness.
If we're serious about putting these things in the driver's seat of a new and automatic science, then we need to know what they're really made of in terms of the ability to create and discover through intuition. Giving shape to the formless, making decisions based on vibes, having "taste"—whatever you want to call it—Can they do it? Do they have the sauce?
Picbreeder, a website where human users collaborated to spontaneously evolve images, is just the sauce-bearing test we need. Here, images were represented as neural networks that could be bred and mutated, with humans playing the role of natural selectors. By design, this interface prohibits the creative baggage of premeditation, of having goals in advance, and demands the artist patiently follow the flow of the work and seize upon serendipitous opportunities when they arise. It's more like catching fish from a stream than drawing a picture. And yet, distributing their work across many sessions, and branching and remixing each other's creations, humans were ultimately able to bend these neural networks into all manner of interesting, evocative, and striking images.
So, can large vision language models do the same? On the blog, we've built an interactive archive viewer that allows visitors to walk through galleries of Picbreeder images created by both humans and AI, and judge for themselves.
Call us old fashioned, but we're pretty sure the human output has something special that the AI can't quite yet replicate. We design a number of evaluation metrics to get at this quality. We ask: "How visually different are the images in the archive? How much do they look like real things? How different are the things they look like?"
The numbers show the humans coming out on top. And looking at the AI-generated archives and lineages, we find traces of an anxious attachment to plans and objectives. Often, even when the AI makes an apparent creative leap—e.g. transforming an image of a hood ornament into a side view of a car—it really stays stuck in place in some broader semantic/thematic space. And that's to say nothing of the handful of archives littered almost entirely with top-down views of soda can pull tabs, or high frequency circular patterns that appear chaotic and uninteresting to us, but apparently scratch some perceptual itch in the agents.
And yet we're optimistic. Though the AI's output is less refined, its movement through the stream of images less graceful and vivacious than our own, what we have here is a plausible model organism of open-endedness. The agents indeed (re)discover distributions of novel and interesting images when left to their own devices. They display a keen eye (even sometimes discovering optical illusions that might slip by a casual glance from a human), and explore persistently under considerable creative constraints. This allows us to model factors that are consequential to such open-ended exploration; i.e. injecting noise into the agents' decision making process, playing with their memory, and seeding them with subtly distinct personalities—all of which can be beneficial in the right doses.
And there's something to be said for searching without objectives. Prior work shows that if we optimize Picbreeder's pattern-producing neural networks to resemble a particular image (say, a skull), these representations will be fractured (meddling with their internal weights will immediately explode the skull beyond recognition), while the same neural image found by humans via open-ended exploration is robust to such perturbations, and even shows meaningful variations across them (e.g. the jaw opening and closing). Our VLM agents also stumbled upon images of skulls. Their representations are not as neatly semantically factorized as those discovered by humans, but neither are they nearly as fractured as those discovered by optimization.
This suggests that if we want to have AI build the next generation of AI, then it will be crucial to let them attack this problem through aimless wandering. Without this freedom, future models will be brittle and myopic; with it, they will have developed a more thorough model of the world, and an improved capacity for the kind of creative insight that is so quietly fundamental to the most meaningful of human endeavors.
If we're serious about putting these things in the driver's seat of a new and automatic science, then we need to know what they're really made of in terms of the ability to create and discover through intuition. Giving shape to the formless, making decisions based on vibes, having "taste"—whatever you want to call it—Can they do it? Do they have the sauce?
Picbreeder, a website where human users collaborated to spontaneously evolve images, is just the sauce-bearing test we need. Here, images were represented as neural networks that could be bred and mutated, with humans playing the role of natural selectors. By design, this interface prohibits the creative baggage of premeditation, of having goals in advance, and demands the artist patiently follow the flow of the work and seize upon serendipitous opportunities when they arise. It's more like catching fish from a stream than drawing a picture. And yet, distributing their work across many sessions, and branching and remixing each other's creations, humans were ultimately able to bend these neural networks into all manner of interesting, evocative, and striking images.
So, can large vision language models do the same? On the blog, we've built an interactive archive viewer that allows visitors to walk through galleries of Picbreeder images created by both humans and AI, and judge for themselves.
Call us old fashioned, but we're pretty sure the human output has something special that the AI can't quite yet replicate. We design a number of evaluation metrics to get at this quality. We ask: "How visually different are the images in the archive? How much do they look like real things? How different are the things they look like?"
The numbers show the humans coming out on top. And looking at the AI-generated archives and lineages, we find traces of an anxious attachment to plans and objectives. Often, even when the AI makes an apparent creative leap—e.g. transforming an image of a hood ornament into a side view of a car—it really stays stuck in place in some broader semantic/thematic space. And that's to say nothing of the handful of archives littered almost entirely with top-down views of soda can pull tabs, or high frequency circular patterns that appear chaotic and uninteresting to us, but apparently scratch some perceptual itch in the agents.
And yet we're optimistic. Though the AI's output is less refined, its movement through the stream of images less graceful and vivacious than our own, what we have here is a plausible model organism of open-endedness. The agents indeed (re)discover distributions of novel and interesting images when left to their own devices. They display a keen eye (even sometimes discovering optical illusions that might slip by a casual glance from a human), and explore persistently under considerable creative constraints. This allows us to model factors that are consequential to such open-ended exploration; i.e. injecting noise into the agents' decision making process, playing with their memory, and seeding them with subtly distinct personalities—all of which can be beneficial in the right doses.
And there's something to be said for searching without objectives. Prior work shows that if we optimize Picbreeder's pattern-producing neural networks to resemble a particular image (say, a skull), these representations will be fractured (meddling with their internal weights will immediately explode the skull beyond recognition), while the same neural image found by humans via open-ended exploration is robust to such perturbations, and even shows meaningful variations across them (e.g. the jaw opening and closing). Our VLM agents also stumbled upon images of skulls. Their representations are not as neatly semantically factorized as those discovered by humans, but neither are they nearly as fractured as those discovered by optimization.
This suggests that if we want to have AI build the next generation of AI, then it will be crucial to let them attack this problem through aimless wandering. Without this freedom, future models will be brittle and myopic; with it, they will have developed a more thorough model of the world, and an improved capacity for the kind of creative insight that is so quietly fundamental to the most meaningful of human endeavors.
The AI Picbreeder Experiment: Can AI agents be creative when nobody tells them what to create?View quoted post on X
Blog: pub.sakana.ai/picbreeder-vlm
In our new #GECCO2026 paper, "In Search of the Ingredients of Open-Endedness: Replicating Picbreeder with Large Vision-Language Models", in collaboration with MIT and NYU, we revisit Picbreeder, a lost website where people collaboratively evolved images without any predefined objective. Users simply selected images they found interesting, allowing unexpected forms such as faces, animals, vehicles, and skulls to emerge gradually across many generations and many different people.
We recreated this process using vision-language model agents. The agents explore a shared archive, choose images to branch from, evolve new candidates, publish their favorites, and evaluate the creations of other agents. There is no target image and no explicit definition of what counts as progress.
The results reveal both the promise and current limitations of AI-driven open-ended discovery.
Compared with humans, VLM agents tend to keep circling back to the same kinds of images and concepts. They repeatedly select similar parents, make smaller conceptual leaps, and often refine an existing idea rather than abandoning it in search of something genuinely unexpected.
However, introducing a diverse population of agent personalities substantially improves exploration. In some runs, diverse agent populations approached or matched the human archive on measures of semantic diversity and produced more balanced evolutionary trees.
We also find intriguing evidence that open-ended evolution can produce more robust representations. A skull evolved by the agents changes smoothly when its underlying neural representation is perturbed, less fractured than a skull directly optimized with gradient descent, although still less cleanly disentangled than one evolved collectively by humans.
But perhaps the most interesting result is the gap that remains.
Humans appear better at turning fortunate accidents into sustained creative discoveries: recognizing when something unexpected is worth pursuing, refining it, and then making a larger conceptual leap. The AI agents often notice interesting patterns too, but are more likely to become trapped in them.
We still do not fully understand what enables humans to navigate open-ended search in this way, or what ingredient(s) current AI systems are missing. For now, the results suggest that there remains something important about human creativity that AI agents have not yet learned to reproduce.
This paper will be presented at #GECCO2026 and is nominated for a best paper award! Please check out the interactive blog and technical paper for more details!
Read our full paper: arxiv.org/abs/2605.23908 🐟