depthfirst

Running a lower-cost model multiple times can match the vulnerability detection rate of frontier models at a small fraction of the cost.

On a 30-task subset of dfbench, five runs of DeepSeek v4.1 Flash found about 71% of known vulnerabilities, up from about 52% with a single run, at $4.24 per task.
Running it 15 times increased recall further to 80%, roughly 13% of Mythos 5’s cost, which found approximately 69% of the vulnerabilities.

Because LLMs are stochastic, they can find different vulnerabilities each time they analyze the same codebase. Combining findings across runs increases coverage, but can also introduce more false positives.

To control that noise, we used our harness to review and deduplicate findings across runs, then filter them by confidence. In separate validation experiments, this approach generally improves both recall and precision.

These results suggest that repeated sampling can improve the tradeoff between coverage, precision, and cost. The gains depend on the model, the number of runs, and how findings are filtered.

Read the research: depthfirst.com/research/bon-v…

We are showcasing our work with @nvidia to post-train Nemotron 3.5 Lightning for defensive cybersecurity at GTC.

As AI makes cyberattacks faster, cheaper, and more common, we need to equip defenders with advanced cyber intelligence that is affordable enough to be always on.

That's what we're building at depthfirst: powerful, specialized security models that defenders can run on their own infrastructure, with full control over their models and data, at a fraction of today's cost.

We're proud to work with NVIDIA to strengthen the U.S. AI cybersecurity ecosystem and help put sovereign security intelligence in the hands of defenders.

Today, we’re excited to introduce Agentic Pentesting on the depthfirst platform.

Agentic Pentesting involves autonomous agents that attack your running application like a team of elite security researchers.

Security teams can:
- Find complex vulnerabilities in systems and running applications, with or without source code access
- Confirm whether findings from code scans and other sources can be exploited at runtime
- Run ad hoc agentic pentests or integrate them into CI/CD for new feature releases
- Create tickets in Jira or Linear, or use depthfirst to autonomously remediate vulnerabilities
- Replay an attack after remediation to verify that the vulnerability has been fixed

We built Agentic Pentesting in close collaboration with our customers. We’re grateful to every security and engineering team that helped shape the product and to the depthfirst team that brought it to life.

Learn more from the links in the comments.

We use Codex and other AI models at depthfirst. They’re useful for code review, and we run our Security Reviewer alongside them to see how each model performs when reviewing the same code.

In one internal test repo, Codex raised 3,519 issues (123 security-related). depthfirst found 869 security issues at a >70% true-positive rate.

We wrote about why security review needs deeper application context, how this context informs every pull request review on the depthfirst platform.

Read more in the link in comments.

Our researchers built a vulnerability scanner that costs two cents per scan and completes a scan in around two seconds.

The scanner is built with Jev, a classifier that outputs probabilities rather than generating tokens. On dfbench v1, our flagship model, it finds nearly as many real vulnerabilities as GPT-5.6 Luna, one of the most efficient models by cost per vulnerability.

This new harness breaks vulnerability discovery into parallel tasks, making it possible to scan every package, repository, or commit quickly and at low cost before escalating the hardest cases to other frontier models on the depthfirst platform.

Read more about how we built this in comments 👇