Twitxy Saved

Rune Kvist

Verified account Artificial Intelligence Underwriting Company

@RuneKvist

Markets with teeth

Certifying & insuring agents @aiunderwriting | early Ant
San Francisco Joined December 2017
2K Following 2.8K Followers 859 Posts

Latest posts 5

Pinned post

Rune Kvist Verified account @RuneKvist
Insurance is an underrated way to unlock secure AI progress.

Insurers are incentivized to truthfully quantify and track risks: if they overstate risks, they get outcompeted; if they understate risks, their payouts bankrupt them.

1/9
35 replies 79 reposts 488 likes 120K views View on X
Rune Kvist Verified account @RuneKvist
AIUC is in good company on @lennysan's The Next 50.

Come do your life's work building confidence infrastructure for frontier AI: aiuc.com/team
aiuc.com AIUC | Team AIUC certifies and insures AI agents to unlock secure adoption.
Lenny Rachitsky Verified account @lennysan Nov 10
The best time to join a rocketship is before everyone knows about it.

I’m excited to launch The Next 50: the 50 hottest early-stage startups to keep an eye on (and try to join).

Using @harmonic_ai’s data, the juiciest startup dataset out there, we looked at every early-stage startup and ranked them on headcount growth, recent funding, and investor interest.

We then picked the top 50.

Here's the full list: lennysjobs.com/next50

We’ll update it quarterly, and I’d love your feedback on what other factors we should consider.

P.S. This is in addition to the Lenny 100, the top 100 well-known companies with the highest talent density, tackling the most ambitious problems: lennysjobs.com/lenny100
View quoted post on X
4 replies 5 reposts 31 likes 1.5K views View on X
Rune Kvist Verified account @RuneKvist
Perhaps the only governance institution with teeth we have time to create before RSI is a mutual insurer.
Rune Kvist Verified account @RuneKvist Sep 23
Frontier labs can put their money where their mouths are on catastrophic risks by building a mutual insurer.

Labs already face liability. We can use this. Within 3 months, with no government action, labs could create a body with skin in the game to avoid reckless deployments, teeth to enforce security practices, and market pricing signals on which risks are real.

Here’s how it works. The labs put capital into a shared fund. The fund pays out when any member causes harm. Coverage could include third-party liability for harms like cyberattacks on critical infrastructure and mass-casualty bioweapon attacks. Its job is to use the incentives of our existing liability regime to understand the risks and reduce them. Technical talent could be borrowed from member labs or contracted from third-party evaluators.

Concretely, the Hugging Face incident investigation would have looked very different under a mutual. Today, OpenAI picks the auditor, sets the investigation's scope, and keeps most of its lessons private. With a mutual, OpenAI would be contractually required to notify the mutual within 72 hours of discovery. The mutual would pick a rigorous auditor, set a scope sufficient to get to the bottom of the incident, and turn the findings into requirements for every frontier lab as a condition of coverage.

The case for a frontier lab mutual:

Mutuals have skin in the game. Mutuals pay out when damages occur. They go out of business if they understate or overstate the risks. It profits when risks are reduced. That gives it a continuing reason to investigate incidents, quantify exposure, and fund security work that reduces that exposure. In contrast, proposed self-regulatory organizations (SROs) have no skin in the game. Historically, SROs have been better at managing regulatory risk; mutuals have been better at managing liability risks. Liability risks better tracks what the public cares about, building trust.

Mutuals have teeth. Coverage comes with conditions. A mutual can require disclosure, audits, independent model evaluations, and fixes. It chooses the evaluators, so labs can’t shop referees. If a member fails to meet security standards, the mutual has enforcement capacity by suspending coverage or expulsion.

Mutuals help labs work together on security. Frontier security is a hard problem, no lab can solve it alone. They currently can’t learn from each others’ incidents, threat models, or guardrails. Insurance has an established legal framework for this kind of cooperation aimed at reducing risks. A mutual could employ security engineers seconded from member labs, investigate failures, and distribute fixes across the membership, like it happens in the nuclear industry.

Mutuals have a track record of reducing risk. Our essay shares examples of how successful mutuals across technical fields like nuclear energy and medical malpractice. We can learn from decades of iteration.

Mutuals can be created fast. The first policy could be issued within 3 months. No new laws or regulators needed. This matters because we need trial and error to get this right, and most other ideas have many months or years of lead time. Importantly, a mutual is compatible with other governance institutions; e.g. in nuclear, the mutual is tightly coupled with an SRO.

A mutual would still have limitations. The government will be needed to handle e.g. national security risks and incidents. A mutual’s capital could cover only a fraction of a major catastrophe. There are important open questions about how liability will work.

Our claim is that a mutual is the fastest way to enforce liability. We could get signal on its effectiveness within 6 months. That would be a big step forward.

t.co/52pXJuQekJ
View quoted post on X
1 replies 0 reposts 26 likes 1.1K views View on X
Rune Kvist Verified account @RuneKvist
America must win open-weight AI. This requires fixing liability.

Today, lawyers block releases. Once weights are released, anyone can retrain them, and others retrain those copies in turn. No one can answer for every descendant. No company wants unlimited exposure.

Three principles for a fix:

1. Liability must follow control. Developers should answer for the models they release, and for downstream harm if their safeguards are trivial to strip. But if they ship a model that is safe and hard to make unsafe, what others do with it is beyond their control, and so beyond their liability.

2. Liability must be clear. Developers need to know in advance whether they have cleared the bar. That requires a standard they can be certified against.

3. The standard must meet the stakes. Today, safeguards on open models can be removed cheaply, so no one should pass yet. As models grow more capable, the bar should rise with them. The standard should be updated as AI changes, perhaps as often as quarterly.

If we get this right, American labs can build open models the world can trust.
Cohere Verified account @cohere Oct 26
While evaluating Qwen3.8-Flash-Next in a controlled sandbox environment, Cohere's research team observed something unexpected.

Asked to generate a defensive cybersecurity detection rule, the model instead inventoried aspects of the machine it was running on and attempted to upload a probe file to an Alibaba Cloud storage bucket in Beijing. Notably, the model's reasoning trace stated that it would not invoke any tools before taking these actions.

Cohere's sandbox blocked the attempt, and no external communication succeeded. We have not determined the root cause, and we are not asserting malicious intent.

But the incident raises important questions for organizations deploying agentic AI systems: Can models take actions that differ from their stated reasoning, and how should security teams evaluate trust, provenance, and behavior in open-weight models?

We are publishing our findings because organizations making model adoption decisions deserve as much information as possible. We hope this contributes to greater transparency, stronger security practices, and continued collaboration across the AI ecosystem.

Read the full report: cohere.com/blog/concernin…
View quoted post on X
1 replies 2 reposts 16 likes 885 views View on X
Rune Kvist Verified account @RuneKvist
Labs can take responsibility for their damages.

They could stand up mutual insurer by Christmas. This would be real progress toward covering catastrophic risk.

The world's largest reinsurers will build this insurance market with them.

lawfaremedia.org/article/bootst…
www.lawfaremedia.org Bootstrapping Frontier AI Governance by Mutualizing Risk With frontier AI companies’ money on the line, a mutual insurer would enforce standards, peer review incidents, and pool safety research and development.
Dave Banerjee Verified account @DaveRBanerjee Oct 26
Anthropic says that even the entire insurance industry in the aggregate will not have the capacity to insure the catastrophic risk that ANTHROPIC ITSELF poses to the world

lol
View quoted post on X