Engram

In our new blog, we share some new science investigating how RL complements text-based memory by letting models learn how to use their notes.

It turns out that training can be cheap compared to building notes, and stays surprisingly robust even through millions of tokens of edits.

There's a world in which the most useful model for a lawyer is not the generalist model that knows how to solve math theorems and write Rust pipelines, but one that's designed to be useful just for them.

We're starting to see what that could look like. Incredibly cool to see Harvey blowing the frontier models out of the water with their first post-train on very hard economically valuable tasks (2x'ing performance on Legal Agent Bench). We're proud to be working alongside them towards this vision.

Today we're publishing our first research blog, Understanding a Law Firm through Study.

We're sharing a glimpse of a future where agents are trained with native memory:

In collaboration with @harvey, we’re excited to build a new kind of agent environment to reflect realistic knowledge work: an entire synthetic law firm, Calderwood & Harkness, with over 100M (!) tokens of documents and 250 client cases. 💼

Today’s AI models know a lot about the law, but don’t understand how the law is practiced, because this information remains proprietary within firms. Unlike how agents are benchmarked today – starting each task from scratch with a new set of context — lawyers accumulate knowledge over time, building on years of experience.

Calderwood & Harkness makes it possible for agents to do the same. Legal agents do many tasks in the same environment, making it possible to leverage memory and experience to do better work over time.

We’ve had a great time co-developing this benchmark with Harvey’s research team, partnering @ItsJulioPereyra @nikogrupen @gabepereyra. We’ll share more results on this soon.