Twitxy Saved

Xingjian Bai

Verified account This account is verified. Learn more

@SimulatedAnneal

Ph.D. student at MIT EECS, working with @phillip_isola and @vincesitzmann. Previous RA at @Oxford_VGG.
xingjianbai.com Joined August 2022
647 Following 1.5K Followers 133 Posts

Latest posts 5

Pinned post

Xingjian Bai Verified account @SimulatedAnneal
1/9 Current optimizers are static during training. What if the optimizer could think and react while the network trains?

We let an agent watch the training dynamics and rewrite the optimizer's source code as training goes: a reactive optimizer.

In every language and image setting we tried that has no tuned recipe, it matches or beats the best recipe one can one-shot, and it guards against loss spikes and overfitting by itself. In the near future, optimization decisions may belong to agents, not humans.

📝 xingjianbai.com/reactive-optim…
💻 Try it on your task: github.com/xingjian-bai/o…
🧵
0:25
79 replies 208 reposts 1.6K likes 225K views View on X
Xingjian Bai Verified account @SimulatedAnneal
OPAL also improves 0th-order optimization! A clean new piece of evidence found by Robert that online reactivity can power all types of optimizers.

“
There is an interesting moment at update 243: The record switches its gradient estimator (Boost → Cut), and the gradients change scale. All three agents had tuned their optimizers on Boost's gradients, and they lost ground right after the switch, up to 0.03 in training loss. OPAL noticed, rescaled the LM head's Adam statistics to the new gradients, and kept adjusting. By update ~620, every run was ahead of the record's optimizer again, and the informed runs stayed ahead to the end.
“
Robert Lange Verified account @RobertTLange Oct 23
I was wondering if OPAL could help the optimizer for the 0-order noise val track.

So I took the (old 10M token) Noise benchmark's best validation-loss record (#14, 3.676) and wrapped its optimizer with OPAL. Every 50 updates, gpt-6.1-sol reads the loss and its own diagnostics, rewrites the update rule in Python, and tests it before it goes live.

Results (seed 0):
- record: 3.6758
- no agent (rerun): 3.6773
- agent, neutral prompt: 3.6791
- agent, prompt informed by the record: 3.6695
- same, xhigh reasoning: 3.6656 (−0.010 vs the record)

The OPAL system prompt mattered: telling the agent what its gradients are, how the record's optimizer came about, and how the end-of-run weight merge works turned no gain into a clear one.

There is an interesting moment at update 243: The record switches its gradient estimator (Boost → Cut), and the gradients change scale. All three agents had tuned their optimizers on Boost's gradients, and they lost ground right after the switch, up to 0.03 in training loss. OPAL noticed, rescaled the LM head's Adam statistics to the new gradients, and kept adjusting. By update ~620, every run was ahead of the record's optimizer again, and the informed runs stayed ahead to the end.

What the best agent changed, keeping the record's ANVIL II update for the weight matrices:
- per-layer rates for the residual mixing scalars, which it halved at the start
- matrix rate −10%, +20% on two query matrices, −15% on the value matrices
- longer momentum averaging (higher β1) for the scalars, head and gates
- the head's Adam rescale at the estimator switch

This is, of course, only anecdotal, and a minor improvement. Still, demonstrates that OPAL doesn't diverge and can mine small gains.

Intuitively, online agent-guided adaptation makes the most sense when hyperparameters/configurations haven't been tuned to death.
View quoted post on X
3 replies 0 reposts 32 likes 1.9K views View on X
Xingjian Bai Verified account @SimulatedAnneal
Sorry again to our infra engineers out there 😶 but changing architectures mid-training (e.g. adding and removing sub-modules) to respond to the fine-grained dynamics inside the network sounds pretty fun to me...
Chidhambararajan R (a.k.a Chidha) Verified account @TheSeriousProg Oct 22
@jayden_teoh_ Bro optimizer perhaps can adapt live. But don't think architecture can adapt mid training
View quoted post on X
2 replies 0 reposts 33 likes 2.7K views View on X
Xingjian Bai Verified account @SimulatedAnneal
Thanks @askalphaxiv for supporting blogs. Our Reactive Optimizers (OPAL) post is now on alphaXiv too.

alphaxiv.org/abs/2610.react…
www.alphaxiv.org Reactive Optimizers We put an agent at the heart of the optimizer. It watches the training dynamics, reacts to them, and rewrites the optimizer’s source code as training goes. When no recipe has been tuned for a task,...
Xingjian Bai Verified account @SimulatedAnneal Oct 21
1/9 Current optimizers are static during training. What if the optimizer could think and react while the network trains?

We let an agent watch the training dynamics and rewrite the optimizer's source code as training goes: a reactive optimizer.

In every language and image setting we tried that has no tuned recipe, it matches or beats the best recipe one can one-shot, and it guards against loss spikes and overfitting by itself. In the near future, optimization decisions may belong to agents, not humans.

📝 xingjianbai.com/reactive-optim…
💻 Try it on your task: github.com/xingjian-bai/o…
🧵
View quoted post on X
0 replies 3 reposts 20 likes 2.5K views View on X