Pinned post
Seven days ago, we started an experiment in Recursive Self-Improvement (RSI).
We wanted to explore a simple question: What happens when we let an AI agent iteratively improve its own harness?
So we let Lobe Agent run for an entire week, using our RSI framework to explore and optimize its own harness.
Today, we're sharing the results.
We evaluated the resulting system on FrontierHarness, using Kimi K3 as the underlying model.
Key results:
• 70.0% pass rate (21/30), ranking #1 among 13 harnesses in our preliminary comparison.
• $1.68 per successful task, compared to $3.47 for Codex and $18.34 for Claude Code.
• 86.2% median cache hit rate, with a median completion time of 7m 19s on successful tasks.
These preliminary results suggest that harness-level optimization may be a promising path toward improving agent performance and efficiency without modifying the underlying model weights.
The comparison includes selected retries and differences in execution environments, and should not be interpreted as an official leaderboard ranking.
We plan to fully open-source the experiment's scripts, execution trajectories, and a detailed technical report to enable reproducibility and further investigation.
Alongside this experiment, we've also been building Awesome-RSI, an open-source research map of Recursive Self-Improvement.
The repository brings together papers, frameworks, benchmarks, and open-source projects spanning model-level and harness-level self-improvement, automated AI research, and safety.
It has already accumulated 80+ commits, and we're continuously expanding and refining the collection.
github.com/lobehub/awesom…
We believe open research, reproducible experiments, and community collaboration are essential to advancing this field.
We welcome researchers and developers to contribute, share their work, and explore RSI together. ❤️
Just try🫱lobehub.com
We wanted to explore a simple question: What happens when we let an AI agent iteratively improve its own harness?
So we let Lobe Agent run for an entire week, using our RSI framework to explore and optimize its own harness.
Today, we're sharing the results.
We evaluated the resulting system on FrontierHarness, using Kimi K3 as the underlying model.
Key results:
• 70.0% pass rate (21/30), ranking #1 among 13 harnesses in our preliminary comparison.
• $1.68 per successful task, compared to $3.47 for Codex and $18.34 for Claude Code.
• 86.2% median cache hit rate, with a median completion time of 7m 19s on successful tasks.
These preliminary results suggest that harness-level optimization may be a promising path toward improving agent performance and efficiency without modifying the underlying model weights.
The comparison includes selected retries and differences in execution environments, and should not be interpreted as an official leaderboard ranking.
We plan to fully open-source the experiment's scripts, execution trajectories, and a detailed technical report to enable reproducibility and further investigation.
Alongside this experiment, we've also been building Awesome-RSI, an open-source research map of Recursive Self-Improvement.
The repository brings together papers, frameworks, benchmarks, and open-source projects spanning model-level and harness-level self-improvement, automated AI research, and safety.
It has already accumulated 80+ commits, and we're continuously expanding and refining the collection.
github.com/lobehub/awesom…
We believe open research, reproducible experiments, and community collaboration are essential to advancing this field.
We welcome researchers and developers to contribute, share their work, and explore RSI together. ❤️
Just try🫱lobehub.com