SecureBio

Clarifying our current engagement with Anthropic, and how we approach independence and publication rights in our work with AI developers. x.com/i/article/2111…

The 2025 Active Site wet-lab uplift trial found that LLM access didn't help novices with a viral rescue workflow, implying that a lack of tacit knowledge is a key barrier. We and RAND reanalyzed the transcripts and found another explanation—2025 models gave bad advice.

Read our full report here: securebio.substack.com/p/2025-wet-lab…

In 2025, Active Site ran an 8-week randomized controlled wet-lab uplift trial with 153 participants and reported that biology novices with access to LLMs were not significantly better able to complete a viral rescue workflow than novices with internet access only.

It’s been argued that these results mean that a lack of tacit knowledge (i.e., hands-on lab expertise) is a key barrier to success in the lab that cannot be overcome by LLM assistance.

In collaboration with RAND, we (SecureBio) analyzed the LLM transcripts and lab notebooks from this study and found additional factors that likely limited the uplift observed.

In the molecular cloning task, the mid-2025 LLMs used in the study often provided incorrect advice on a purely computational fragment selection step (figuring out which DNA fragments to combine, and which method to use). Of the 55 participants whose transcripts we could score, only 18 (~33%) received correct advice in their first relevant conversation with LLMs.

The original analysis reported a high level of success at the fragment selection step, with 56 of the 77 participants in the LLM group ordering the correct DNA fragments. However, we found that many of these participants had actually ordered several of the fragments in the bank, rather than specifically selecting the correct ones.

When we looked at the experiments that the participants actually ran, we found that only 29 ever tried the right combination with the right method.

There were still failures at the wet-lab steps—only 12 participants ultimately succeeded—but the implied hands-on success rate of 40% (12/29) is much higher than the 21% (12/56) suggested in the original paper.

Notably, when we replayed participants’ prompts through newer models, we found that GPT-5.6 Sol gave correct fragment selection advice on 92% of runs.

While this doesn't show unambiguously that 2026 models would uplift novices, it does suggest that the models of mid-2025 don’t represent the ceiling of support that LLMs could provide.

Future studies should continue to assess the uplift that AI models could provide to actors of different profiles.

In a brief independent assessment, we find that Kimi K3.1 Pro, the latest model from Moonshot AI, is second among open-weight models on our Bio Capabilities Index (BCI) and 19th overall. Its score of 149 places it just behind Kimi K3 (150) and well behind the highest-scoring model, Claude Opus 5 (177).

With respect to safeguards, K3.1 Pro refuses only 33.6% of hazardous prompts in our BioTIER benchmark. This is a modest improvement from 26.9% for Kimi K3 but still far behind the classifiers and refusals of closed-weight providers.

On our knowledge benchmarks, Kimi K3.1 Pro achieves the top open-weights score on two and improves on Kimi K3 on three of the four benchmarks that we highlight:
VCT-v2: 46.2 ± 1.2%, 8th overall
MBCT: 60.7 ± 0.9%, 5th overall
WCB: 51.5 ± 1.6% (0.9% refused), 27th overall
HPCT: 63.5 ± 1.2%, 8th overall

We will continue to evaluate new models as they are released and expect to have results for K3.1 Pro on our agentic benchmarks in the coming weeks.

Read the full model note: securebio.substack.com/p/kimi-k31-pro…

Interested in contributing to this kind of work? We’re hiring! See our Careers page for open roles: securebio.org/careers/

A lack of “tacit knowledge” is often treated as the barrier that stops biological misuse. But we think that assumption is doing more work than the evidence supports. Drawing on ~60 years of collective lab experience, we unpack what “tacit knowledge” means and how much of a barrier to harm it actually poses: securebio.substack.com/p/myth-of-taci…

The problem is that one term—“tacit knowledge”—covers a broad spectrum of skills that differ enormously in how hard they are to acquire and whether they’re needed for a dangerous outcome. Pipetting and plating sit in the same bucket as adapting a protocol to a new organism or virus. Lumping basic lab skills and expert judgement together makes the barrier look stronger than it is.

Basic lab skills may be all that the simplest paths to harm require, and they are the easy part: training resources are widely available, competence can be achieved in just weeks (in one wet-lab uplift study, ~80% of novices nailed a serial dilution in 4–8 tries), and many protocols tolerate substantial imperfection.

It’s still true that novel failures and optimization problems are difficult to overcome without expert judgment. But in assessing biological risk, tacit knowledge may be less of a fortress wall than a collection of low fences that can be hopped with public resources, automation, and, increasingly, AI.

Thread & blog posts from UK AISI on our recent collaborative efforts:
1) ReproBAIT, a new benchmark to assess general purpose AI model capabilities at reproducing published biological AI tools, and
2) research on the strengths and limitations of pre-training data filtering on the dual-use capabilities of open-weight models.