Twitxy Saved

Ian Osband

Verified account This account is verified. Learn more

@IanOsband

Research scientist at DeepMind (ex OpenAI, Stanford) working on decision making under uncertainty.
ianosband.com Joined July 2012
376 Following 11K Followers 811 Posts

Latest posts 5

Pinned post

Ian Osband Verified account @IanOsband
Assembling a team at DeepMind in London.

Scaling up RL for post-training is working, but right now it's still mostly hacks and dark arts (pretraining circa 2019).
Pre-training wasn't always scaling laws and log-log plots; someone had to find the simplicity.
We aim to do the same.

If you're interested in doing things right in a research-first environment that scales all the way, please apply: job-boards.greenhouse.io/deepmind/jobs/…
20 replies 66 reposts 993 likes 159K views View on X
Ian Osband Verified account @IanOsband
Wrote a little Colab for "Planning to Learn".

Runs in ~3 mins on a free T4 GPU:
- Exact PG loses to cross-entropy, even on accuracy.
- The horizon loss beats both.
- Planning CE even beats CE on its own NLL metric.

Nothing hidden... Run it yourself!

colab.research.google.com/drive/1nuACbgy…
colab.research.google.com planning_to_learn.ipynb Colab notebook
Ian Osband Verified account @IanOsband Oct 5
In LLM post-training, policy gradient is "the RL loss".

When it fails, we blame the RL: exploration, credit assignment, sampling noise.

Image classification has none of these, and the policy gradient is exact. Following it is terrible...
ImageNet, 4% vs 62% for cross-entropy.
View quoted post on X
8 replies 9 reposts 103 likes 7.1K views View on X
Ian Osband Verified account @IanOsband
In LLM post-training, policy gradient is "the RL loss".

When it fails, we blame the RL: exploration, credit assignment, sampling noise.

Image classification has none of these, and the policy gradient is exact. Following it is terrible...
ImageNet, 4% vs 62% for cross-entropy.
11 replies 25 reposts 252 likes 23K views View on X
Ian Osband Verified account @IanOsband
Agi achieved 1993... Thank you Jurgen

Even before Gary pioneered Neuro symbolic systems
Jürgen Schmidhuber Verified account @SchmidhuberAI Sep 11
Thanks! 😀 Indeed, I solved neural self-reference (NS) in 1993 [1][2].

[1] JS. A self-referential weight matrix. ICANN 1993, Amsterdam, p 446-451
[2] K. Irie, I. Schlag, R. Csordas, JS. A Modern Self-Referential Weight Matrix That Learns to Modify Itself. ICML 2022
View quoted post on X