Twitxy
Saved
@jackm2003's profile picture
View on X

Jack Miller

Verified account This account is verified. Learn more

@jackm2003

🦘/acc + CTO of @HelmGuard. I sometimes post about deep learning.
London, England helmguard.ai Joined October 2018
213 Following 247 Followers 633 Posts

Latest posts 5

Jack Miller Verified account @jackm2003
There's a theory of AI adoption that it will come in two waves. In the first wave (phase 1), businesses take an existing process, and automate parts of it with AI. In the second wave (phase 2), people will replace the existing process with something adapted to the new paradigm.

What will this look like in cybersecurity? Do we have a sense of what a phase 2 use case might be?

I think a very good candidate is third party compliance and security.

Today AI is being used in the space to speed up an existing system: you design an agent to be 10x faster and cheaper at extracting claims from a SOC2 report or reviewing the outputs of a penetration test.

I think there is an entirely new system that can be built around the idea of "stateless auditing" or as we call it at @HelmGuard ephemeral verification. This leverages the fact that AIs can forget the information they see. They can verify facts about data without revealing that data to the verifier.

I think that in the future, if you want to verify whether a supplier you are working with is secure to your standard, you won't ask them for a SOC2 report. You'll send a specification of what you consider secure to a trust agent in their environment which will verify those claims for you and return a yes or a no.
Jack Miller Verified account @jackm2003
One underrated aspect of critical infrastructure security is the susceptibility of these orgnaisations to supply chain attacks. If I were a threat actor, I wouldn't look to compromise a public healthcare provider or energy grid operator directly, I'd infiltrate their supply chain.

What makes this easier than in most other sectors:
1. Critical national infrastructure often has a public record of all suppliers used (this is usually to keep procurement fair)
2. Suppliers are typically legacy companies with very poor security
3. The number of suppliers is enormous

Instead of going after infrastructure where there have been heavy investment in internal security, I'd go after the supplier they've been relying upon for 20 years that hasn't gone through a security due diligence process.
Jack Miller Verified account @jackm2003
As we contemplate a slow-down on model progress. I think it's important to ask what a pause would mean for cyber defence.

As the co-founder of a cyber defence company, I personally think there's years of progress we could make in elicitation and diffusion if we froze at Fable-level capabilities.

For example, looking at one type of work security teams need to do on the regular: evidence collection and control mapping. Organisations have a bank of controls that they use to regulate what is permissible e.g. "all privileged accounts must have MFA" or "all development work must be done on a remote VM with a secure OS". Practitioners then need to manually assess whether this control is enforced across all of the relevant assets an organisation owns or across all of the people that work there.

Today there are too many assets, people and programs for these teams to assess, so they sample a controls' effectiveness or use deterministic rules. However, this work is relatively easy for an AI model, it involves some multi-modal understanding but primarily reasoning across textual inputs. On our evals, this sort of work is easily handled by the Opus 5 and Terra 5.6 class of models.

It would be a phase change for cyber defence if controls were assessed at a population level not by sampling. That is by looking at everything an organisation is doing.

This is just one example, but there are countless others where a pause wouldn't impact progress.
Jack Miller Verified account @jackm2003
I’m quite into this idea
Tom Reed Verified account @mentalgeorge Apr 28
Wacky but secretly-good idea that follows from taking the whole "AI constitutions" premise seriously:

There should be "courts" to make rulings on how the Constitution/Spec applies to contentious instances of model behaviour.

Users should be able to report + contest costly refusals through a button in the app - most of these are ofc auto-resolved with AI, but the interesting edge cases bubble up through a hierarchy of courts, with the most contentious of all being resolved by a "supreme court".

Many advantages to this approach:

1. Claude's ability to apply the constitution to a given situation will be massively, massively improved by having a rich body of precedent to contemplate and refer to. Such a body of precedent makes the constitution a lot "thicker" - it could also be open-sourced to improve the (currently somewhat disappointing) transparency into the Constitution, allowing users to know what to expect from Claude.

I imagine that Amanda, Joe et al currently produce such "precedent" for Claude with synthetic data and galaxybrain theorycrafting, but I'd happily bet that the real world is a better, richer source of edge cases than anything even they could come up with.

2. This could be a really, *really* interesting way to get the "democratic input" into AI constitutions that everyone keeps clamouring for. Usually these proposals end up as uninspired calls for using surveys and focus groups in the drafting stage of the Constitution, which I think is a fairly limited way to think about the strengths of liberal democracy.

On this "courts for the spec" proposal, you could imagine opensourcing stages of the judicial process in wide variety of ways. One thing you could do is to crowdsource amicus briefs (or perhaps even the whole case!) for petitioner or respondent. I feel like there's a promising Pettit angle to this.

3. Discursive and argumentative traditions (and not just surveys or isolated technocrat drafting) have a good track record for being the means by which we as humans resolve these kinds of problems, so it just makes sense to get this going for AI. I particularly think that debates are likely to be much richer when they are about *specific* instances of model refusals than just vague, open-ended discussions about how AIs should be governed. It's also predisposed to iteratively co-evolve with the pace of technological change far better than any one-and-done philosophising would

4. There may even be some kind of "separation of powers" argument to make here, insofar as the "legislature" drafting the spec/constitution is distinct from the "court" ruling on how it applies to a particular case. More applicable to OAI than Anthropic (which is more generally comfortable with moving past traditional liberal principles) ofc.

5. Finally, and most obviously, there *should* be a way to contest model refusals!!!

If we all take seriously the idea that agents will become some large % of the human economy, then an individual unjust refusal could be absurdly costly.

I think there should be a transparent, participatory way to contest this such that we are not at the mercy of the AI conglomerates. I cannot think of a better way than to borrow from the law. Perhaps I'm being too Anglobrained with this, but I just think the law and courts pareto-mog almost all other forms of value-resolution we homo sapiens have ever come up with
View quoted post on X