> A common idea floating around is that LLMs will absorb harnesses and make them obsolete.
There is no way enough people believe this for it to be "common".
There is no way enough people believe this for it to be "common".
Harnesses used to be a crutch for weak LLMs. In 2026 and onward, with stronger models, harnesses now offer a mechanism for composing an LLMβs existing and new capabilities to generalize well beyond their training environments.View quoted post on X
In our new paper, LM Harnesses are Compositional Generalizers, we explore how various harness abstractions enable LM systems to generalize over task length, task strategies, recursive task depth, and more through composing existing LLM capabilities. These abstractions provide a means for more data-efficient training, especially over long tasks, by keeping trajectories locally in-distribution (LID) β i.e., shaping all individual LLM calls to be in-distribution.
For tasks that take a very long time to roll out, this implies that we can even perform surrogate harness training, meaning we train an individual LLM call instead of doing the very long harness rollout, using purely synthetic tasks, with fake tool calls + sub-agents!
A common idea floating around is that LLMs will absorb harnesses and make them obsolete. We disagree, and argue that we havenβt sufficiently explored what harnesses can provide.
The full paper, logs, codebase, and data are available: alphaxiv.org/abs/2611.harneβ¦
w/ @SOURADIPCHAKR18 @lateinteraction