Latest posts 5
For any serious engineer, this shouldn鈥檛 really come as a surprise. Opus 5.5 is just on another tier
Nice DeepSWE score! Does it survive a lowercase filename?
Introducing MaxxBench: 89 DeepSWE tasks with small changes to the instructions or starting code. Same engineering problem, slightly different requirement.
Models clustered around 74% on DeepSWE end up 23 pp apart.
Good at coding, or good at DeepSWE? 馃УView quoted post on X
A year ago, announcing our $61M raise, I wrote down the problem I most wanted to hire for: an AI system that hallucinates less than 0.01% of the time.
In May we published under 1%, down from 4-5%, across 1.2M live voice turns. Still 100x away.
The last 1% is an entirely different problem. Questions the knowledge base doesn't cover, instructions that can honestly be read two ways. 20-minute calls where early turns get summarized and a key fact gets lost.
A faster detector doesn't fix that. So we're feeding every caught hallucination back upstream: instruction rewrites, knowledge base audits, RL on the detector.
Year two now is about that last 1%
In May we published under 1%, down from 4-5%, across 1.2M live voice turns. Still 100x away.
The last 1% is an entirely different problem. Questions the knowledge base doesn't cover, instructions that can honestly be read two ways. 20-minute calls where early turns get summarized and a key fact gets lost.
A faster detector doesn't fix that. So we're feeding every caught hallucination back upstream: instruction rewrites, knowledge base audits, RL on the detector.
Year two now is about that last 1%
Alibaba's sales agent lifted conversion 15.8% in a live A/B test across 200,000+ merchants, without touching the model's weights.
It does outreach on 1688, Alibaba's B2B wholesale marketplace, where a deal usually takes several conversations. Most sales agents start every call from scratch.
SPAR treats each selling strategy as a hypothesis. It starts at 50/50 and its score moves toward its real success rate as results come in, so two lucky wins don't make it a favourite. If a merchant walks away because their budget got cut, an LLM marks that as external and it doesn't count against the pitch.
ExpeL, an earlier method that saves lessons as free-text notes, got +1.1% in the same test.
It's very similar to what we launched with Scout in June. You tell it the KPI you care about, it learns from every conversation, tests each change and keeps improving that number.
Optimising on business KPIs is the future for agents.
giga.ai/scout
It does outreach on 1688, Alibaba's B2B wholesale marketplace, where a deal usually takes several conversations. Most sales agents start every call from scratch.
SPAR treats each selling strategy as a hypothesis. It starts at 50/50 and its score moves toward its real success rate as results come in, so two lucky wins don't make it a favourite. If a merchant walks away because their budget got cut, an LLM marks that as external and it doesn't count against the pitch.
ExpeL, an earlier method that saves lessons as free-text notes, got +1.1% in the same test.
It's very similar to what we launched with Scout in June. You tell it the KPI you care about, it learns from every conversation, tests each change and keeps improving that number.
Optimising on business KPIs is the future for agents.
giga.ai/scout
People pay for gallon of oil not how much useful energy it produces
This is the equivalent of counting the atoms in a barrel of oil rather than measuring the useful energy it produced.View quoted post on X
