Twitxy Saved

David Lindner

@davlindner

Making AI safer @GoogleDeepMind
London, UK davidlindner.me Joined April 2012
347 Following 1.8K Followers 215 Posts

Latest posts 5

David Lindner @davlindner
Great to see our AI control team sharing more of their thinking and plans for how to build safeguards for potentially misaligned AI agents. Worth a read if you're working on related research!
Mary Phuong @MaryPhuong10 Jun 18
We're releasing the GDM AI Control Roadmap -- our plan for building internal security against potentially adversarial AI agents, as they grow harder to oversee and contain.

Paper: storage.googleapis.com/deepmind-media…
Blog: deepmind.google/blog/securing-…

🧵👇
View quoted post on X
1 replies 0 reposts 6 likes 1.1K views View on X
David Lindner @davlindner
We looked at exploration hacking, a much-talked-about safety problem with so far ~no empirical work

Bad news: we can make LLMs strongly resist RL elicitation

Good news: we had to try pretty hard and it's easy to detect

Excellent work led by @BraunJoschka @eyonjang @DamonFalck
Joschka Braun Verified account @BraunJoschka May 4
RL assumes that LLMs explore well during training. What if they choose not to?

In our new ICML paper with @GoogleDeepMind, we train LLMs that strategically resist RL capability elicitation by under-exploring.

We study this threat model, called exploration hacking.
View quoted post on X
2 replies 7 reposts 48 likes 4.9K views View on X
David Lindner @davlindner
Training against the CoT might break CoT monitoring. But not all "training against the CoT" is equally bad

We show some rewards are worse than others and propose a framework for categorizing them

This helps anticipate which training might hurt CoT monitoring!
Max Kaufmann Verified account @Max_A_Kaufmann Apr 1
Is training against the CoT always bad?

RL training can lead to obfuscated CoT making it difficult to 'read an LLMs thoughts'. How can we predict when obfuscation occurs?🤔

Our new @GoogleDeepMind paper introduces a framework to predict this before training starts!
View quoted post on X
1 replies 2 reposts 19 likes 1.8K views View on X
David Lindner @davlindner
We proposed a formalization of why current LLMs cannot do a concerning amount of opaque reasoning

This is super useful for thinking about when we can rely on chain-of-thought monitoring and when it might fail

But we can also **measure** the opaque depth of any neural network!
Rohin Shah @rohinmshah Mar 23
"Just read the chain of thought" is one of our best safety techniques. Why does it work? Because models can only think opaquely for a short time, long thinking must be transparent

Can we quantify this? Yes! In our new paper, we show how to measure "time" for arbitrary networks.
View quoted post on X