Judgment Labs

Built for long-horizon agents.

Monitor agents against your own criteria.
Search and investigate patterns at scale.
Benchmark improvements and regressions before rollout.

judgmentlabs.ai

Judgment Labs -> Judgment Slabs

Kicked off @aiDotEngineer Code Summit on the wall 🧗

Thank you to everyone who came out, and to @cognition and @GoogleDeepMind for climbing with us.

Should we run this back 👀 👀 👀

Introducing the Judgment Slack Agent

You can now ask Judgment about ANY part of your JL workspace (data analysis, monitoring, issues, offline tests) without leaving Slack.

Just tag @Judgment with a request like "send me an issue report on our support agent every morning at 9am" and JL will post new issues to your channel, ranked by severity. You can also use it to update your judges or set up new tests, right from the thread.

Interacting with JL and catching silent agent failures has never been easier. Start for free today!

Introducing Judgment Actions

Your judges can now take action automatically when it flags a behavior.

For example, if you’re monitoring a customer support agent for signs of user frustration, your trace judge can alert your team in Slack or by email and save the flagged trace to a dataset for later review. Here's what that alert on a customer support agent looks like.

Is your Jev-as-a-judge, LLM-as-a-judge, or agent-as-a-judge checking for what you care about? Is it making the evaluation decisions you would?

You can now create and align judges with the Judgment Agent. Start by describing what you want to catch, or bring your own rubric.

The hardest part of judge alignment is finding examples that expose where your rubric falls short. The Judgment Agent searches your real traces for challenging cases and tests the rubric against them, surfacing ambiguities for you to review.

Agree or disagree with the verdicts, and explain what it missed. The agent uses your feedback to refine the rubric and align the judge’s decisions with your own before creating it.