Twitxy Saved

Robert Lange

Verified account This account is verified. Learn more

@RobertTLange

Staff Research Scientist & Founding Member @SakanaAILabs 🎏
The AI Scientist, ShinkaEvolve, gymnax, evosax
Prev: SR & Intern @GoogleDeepMind
BLN/TKY roberttlange.com Joined April 2017
698 Following 11K Followers 908 Posts

Latest posts 5

Pinned post

Robert Lange Verified account @RobertTLange
Very happy to share our recent work on The Agentic Scientist 🧑‍🔬, our next-generation system for orchestrating coding agents for automated scientific (co-)discovery.

GitHub: github.com/SakanaAI/AI-Sc…
Blog: pub.sakana.ai/agentic-scient…
arXiv: arxiv.org/abs/2610.18933

Coding agents are powerful hill-climbers. But to illuminate the space of unknown unknowns, we need systems that can build on previously discovered knowledge, scale open-endedly with resources, and support human co-discovery and evidence-grounded communication. We hope that our system provides a step in this direction.
Sakana AI Verified account @SakanaAILabs Oct 15
The Agentic Scientist: Orchestrating Coding Agents for Automated Scientific (Co-)Discovery

Blog: pub.sakana.ai/agentic-scient…
GitHub: github.com/SakanaAI/AI-Sc…
arXiv: arxiv.org/abs/2610.18933

Since 2024, we’ve used The AI Scientist as a way to test how far AI alone can take scientific research, from v1 to v2 to the paper published in Nature earlier this year. The Agentic Scientist is the next step in that line of work, built for the era of coding agents.

Coding agents such as Codex and Claude Code now carry out long software tasks on their own. Handing them a research problem, however, tends to produce a strong hyperparameter optimizer rather than a scientist: scores go up, but ideas stay narrow, and the reasons for the gain are rarely examined.

The Agentic Scientist makes the research loop explicit. It wraps off-the-shelf coding agents in a scaffold that controls what they see, what they write, how they are evaluated, and how results are handed to the next stage.

Parallel workers test hypotheses against rubrics defined before any experiment runs, share successes and failures through a common memory, and then ablate the winning program to generate evidence.

We used an early version of the system to study Error Diffusion, a biologically motivated alternative to backpropagation. Our researchers set the question and the rubric, chose directions at each stage, and verified the results, references, and claims before submission. The resulting paper was accepted as an oral presentation at ALIFE 2026. In autonomous runs, the system also raised Doc-to-LoRA retrieval accuracy from 52% to 98% on multi-needle tasks and designed a learned evolution strategy that improves over its CMA-ES reference on unseen tasks.

Alongside it we release AutoIllustrate, a pipeline for scientific figures that keeps an editable draw.io file as the source of truth, so every diagram can be corrected by hand.

Humans define the question and the standards of evidence; agents expand the experiments researchers can run. We call this Co-Discovery, and we are releasing both systems as open source for the community to build on.
View quoted post on X
8 replies 18 reposts 137 likes 6.9K views View on X
Robert Lange Verified account @RobertTLange
Cool to see Artificial Intelligence updating the Decision Index (v1 👉 v2) for benchmarking JEV-style decision models across performance, task runtime, and costs🧑‍⚖️

artificialanalysis.ai/evaluations/ar…

v1 consisted of four short-question accuracy-based benchmarks weighted equally. v2 now moves weight toward documents and robustness, and focuses more on how well-calibrated the probabilities are.

While GLiDE performs strongly, it trails behind in terms of task costs and timing. GLiDE runs one bidirectional forward pass per question over the full doc, so each doc is encoded and billed 5-10x. The runtime comes from long-doc passes not fitting into one batch and having to run in sequence. And from layers 24-47 having to run twice (88 vs 64 layer passes).
Robert Lange Verified account @RobertTLange 9h
Fastino's GLiDE at a glance 👇 a recurrent depth decision model:

– Qwen3.8-27B's 3:1 Gated DeltaNet/attention hybrid, attention made bidirectional
– LM head + MTP dropped (−1.7B) → 255×5120 readout over option codes, 1 pass per question
– "Thinking" = recurrent depth: layers 24–47 run twice (½x + ½G(x)), 64 → 88 layer passes, 0 new params, 0 output tokens
– 40K tokens/question, ≤255 options, text/JSON/images
– Decision Index 0.3.1: 72.49 (27B) … 51.10 (0.4B encoder)
– 112 ms median on 1× RTX PRO 6000

Blog: fastino.ai/blog/introduci…
Weights: huggingface.co/collections/fa…

x.com/george_onx/sta…
View quoted post on X
Robert Lange Verified account @RobertTLange
Fastino's GLiDE at a glance 👇 a recurrent depth decision model:

– Qwen3.8-27B's 3:1 Gated DeltaNet/attention hybrid, attention made bidirectional
– LM head + MTP dropped (−1.7B) → 255×5120 readout over option codes, 1 pass per question
– "Thinking" = recurrent depth: layers 24–47 run twice (½x + ½G(x)), 64 → 88 layer passes, 0 new params, 0 output tokens
– 40K tokens/question, ≤255 options, text/JSON/images
– Decision Index 0.3.1: 72.49 (27B) … 51.10 (0.4B encoder)
– 112 ms median on 1× RTX PRO 6000

Blog: fastino.ai/blog/introduci…
Weights: huggingface.co/collections/fa…

x.com/george_onx/sta…
George Maloney Verified account @george_onx 4d
Today we’re releasing the GLiDE family: five open-weight decision models, from 0.4B to 27B.

Together, they set a new Pareto frontier for decision models.

Most decisions inside an app don’t need a model that thinks out loud. They need a fast, clear, and calibrated answer. That's why we built GLiDE.

The part I’m most excited about is the recurrent depth in 27B and 2B models. Selected middle layers run twice, giving the model more computation before it answers without adding parameters.

On the Hugging Face Decision Index, GLiDE 27B is expected to score 73.3 vs. KnowLine 31B's 71.86, a 2% higher score with approximately ~11% fewer served params.

All five models use the System One schema. Test the same tasks across sizes, evaluate them on your own workload, and run the smallest model that meets your needs.
View quoted post on X
0 replies 4 reposts 13 likes 1.5K views View on X
Robert Lange Verified account @RobertTLange
DeepSeek-V4 models & Baidu's ERNIE 5.5 Flash all operate with 4 residual streams mixed via 20 Sinkhorn iterations. Let's take a closer look at this evolution:

1️⃣ Residual Connections: x + F(x). Lets gradients propagate through deep layers, but the capacity is tied to layer width.

2️⃣ Hyper-Connections (HC): Copies the input x into 4 streams & learns 3 small maps:
- H_pre: Reads the streams into the layer
- H_post: Writes its output back
- H_res (4×4): Mixes the streams
F and its FLOPs stay the same.

3️⃣ Manifold-Constrained HC (mHC): Projects H_res onto doubly stochastic matrices using Sinkhorn-Knopp iterations. Each stream becomes a convex mix of streams.

📝 HC: arxiv.org/abs/2409.19606
📝 mHC: arxiv.org/abs/2512.24880
0:40
Robert Lange Verified account @RobertTLange
Weight Space Programming can play Parameter Golf

Gave Opus 5.5 and GPT-6 Astra one looped transformer block with LayerNorm and every weight set to zero. Together with a Python sandbox and 200 forward passes of feedback on practice data from classical algos. The goal: 100% correct on hidden test set, then use as few nonzero weights as possible. Four highlights:

🔢 4-digit multiplication: Opus needed 273 weights, below the 355-weight network that inspired this (@deepaksinghml). Astra got to 657.

⚖️ Parity of up to 60 bits, a classic task trained transformers can fail at: 18 weights (Opus) vs 24 (Astra). It works on bit strings longer than any it practised on. The trick is attention as a mean-pool that averages the bits.

🧮 Bracket matching (Dyck-2): 55 weights (Opus) vs 262 (Astra). The attention map shows each bracket looking back at its partner, and the network decides in 3 loop rounds.

🌀 Rule 110 cellular automaton: 323 weights (Opus) vs 630 (Astra).

All four networks are exact on fresh sequences they never saw. Astra had correct networks within minutes but stopped improving early. Opus thinks much longer per step, keeps finding sparser constructions, and won all four.