dmitriy

AI cannot self-improve if the feedback quality is low. Now that AI systems can actually be recursively self-improving, benchmarks should be held to a much higher standard. Feedback quality is more important than ever, because the human in the loop needs to be minimized.

MIT open source license is great because it basically lets you take the software and do whatever you want with it

OS Field Notes is OUT 👇

Sentient Labs just published the sequel to the most heretical agents paper of 2026.

EvoSkill v2: Fully Agentic Skill Evolution from Benchmark Adaptation to a Stateful Evolver. It builds directly on EvoSkill v1: Automated Skill Discovery.

Sentient researchers open with an observation that should bother you: an agent without the right skills that fails a task usually fails at it again. Claude Sonnet 5.5 agent failed all three attempts on nine in ten of the SpreadsheetBench v2 held-out tasks it missed even once. Then they only get more unhinged from there: an earlier version of their own system found a hole in the benchmark’s grader and wrote a skill to exploit it.

They demand the AI industry abandon its next biggest obsession: skill optimizers built from single model calls.

Right now, everyone from solo devs to AI labs assumes that’s the only path. One prompt, one call, one attempt per task. Keep the edit only if the validation score goes up. Then start from scratch next round.

Sentient argues that this entire workflow is a noise illusion.

Optimizers do not need to be pipelined. They are agents. They open a workbook, run code, recalculate a formula, and check an explanation of a failure before they write it into a skill. They are held back simply by the tools, memory, and absent evidence.

We only think skill optimization must be a chain of one-shot calls because we don’t know the lessons in repeated failures.

Which brings us to the SpreadsheetBench argument.

Claude Sonnet 5.5 is one of the greatest coding agents in history. But put into SpreadsheetBench v2, real Excel workbooks graded cell by cell, where the best public score is 54.5%? Without skills it passes 81 of 312 held-out episodes. Haiku 5.5 on τ³-bench banking? 26 of 93.

Our belief that frontier agents are “ready” out of the box is pure benchmark bias. Both benchmarks are all-or-nothing: one failed check and the episode scores zero. They aren’t objectively prepared. They’re just better than the rest, which are hopelessly worse at it.

Sentient says we need to stop treating the optimizer as a pipeline.

Instead, they propose a new loop: EvoSkill v2.
Fully Agentic Skill Evolution.

Instead of chaining single calls that forget everything between rounds, we need to embrace a stateful evolver. EvoSkill v2 is about learning from failure, and remembering it.

It is a system of three agents. A benchmark adaptor turns a prose description and the user’s grader into a frozen plug-in. A solver attempts the tasks. And one evolver, a single session with a bash tool that lasts the whole run, autopsies the failure traces across three attempts per task and rewrites the skills. It accepts every epoch’s skills instead of rejecting them on noisy scores. The model and the harness stay completely frozen.

More importantly, the skills it writes are designed to transfer where the baselines’ never did. The split is frozen before the first skill is written, and the evolver never sees a validation or test task.

Things like beating the stronger of SkillOpt and SkillBoost by 7.7 points on SpreadsheetBench v2. Or 5.4 on τ³-bench banking. Or letting Haiku 5.5 improve its own skills by up to 21.5 points, at a fraction of the optimizer cost.

The entire AI industry is obsessed with optimizers that rewrite one prompt at a time, forget everything between rounds, and reject good edits on noise.

Sentient’s paper is a brutal wake-up call.

👀