Max Fu

Jay Chooi @ CoRL 2026 Verified account @chooi_jeq · Sep 5

GPT-6 Astra scored 95% on a robot control task, up from Fable 5.1's 40%, with 6.2x fewer output tokens at 2.3x lower cost. 🧵

View quoted post on X

Pete Florence Verified account @peteflorence · Aug 19

Ever since I started working on robot foundation models a handful of years ago, the broad ability to one-shot in-context learn has been the single most vivid goal in my mind along the long road ahead. It’s something @andyzengineer and I have especially been thinking together on for several years now, and something shared by the whole team at Generalist as a major goal.

But even before anybody started talking about robot foundation models, too, the capability this enables, i.e. generalized one-shot learning, has been both incredibly concrete and elusive. In terminology that I think many other people have come up with as well, in grad school we used to talk about a “Ctrl-C-Ctrl-V” type of capability. See something, and have the robot do it too. This same type of idea inspired the name of the 1970 MIT copy demo (people.csail.mit.edu/bkph/phw_copy_…).

The thing is, it sounds simple, but is incredibly hard since the world is never quite the same when it got copied and where you want to paste it. The real world can be hard to predict and is full of variation. To do this, you need strong generalization, and it needs to be acquired in a single example. Doing this over a wide range of tasks, especially for dexterous tasks, is hard mode.

Lots of the components of the idea of making this all happen have been there for a long time. As an example, this 2017 NeurIPS paper “one-shot imitation learning” arxiv.org/pdf/1703.07326 has excellent vision, with ambition well beyond what was achievable at the time, and although it’s not referred to as “in-context learning” since it was pre-Transformer, it actually uses attention to condition on a single demonstration.

And now, many things have happened since early 2017, including the broadly celebrated arrival of one/few-shot in-context learning in language models in 2020.

This new model GEN-1.5 takes in everything we have built and learned over the past couple years at Generalist. It has been training for 8 months. It has taken an incredible amount of commitment and grit from the whole team to get here. The level to which this model has survived many surgeries has continued to surprise me. And its capabilities have continued to surprise as well. We found compositional generalization on Friday. We found sim2real prompting earlier last week. We filmed the contiguous uncut videos of live prompting yesterday.

To be clear, the success rates are modest, and there’s still a long way to go. But now I have definitely seen a ~decade-long imagination come into the real world.

View quoted post on X

Stephen James Verified account @stepjamUK · Aug 6

A wrist force sensor fires at 100Hz. The policy only ever sees it at 30Hz, downsampled to land on the same control step as the camera and the joint state.

That's not a bug, it's the whole point, and it sits inside a bigger pattern in VLA research this year. Every major release has been Markovian at its core, mapping the current frame straight to the next action. The fix everyone reaches for is more vision: more history frames, longer image context.

FM-VLA makes a clean case that the fix is the wrong channel for a whole class of tasks. Press a button three times and stop. A camera watching that has almost nothing to work with, the scene barely changes between press one and press three. Force doesn't have that ambiguity problem. Each press is a sharp, distinct spike in the wrench signal, whether or not the camera noticed anything at all.

So FM-VLA doesn't add more frames. It compresses the wrench history into eight tokens with a VAE, pretrained purely on reconstructing force signals, frozen before it ever touches the policy, then hands those tokens to the action expert alongside a short window of joint state. That's the entire memory system.

Averaged across three contact-rich tasks, FM-VLA hits 83.3 percent success against 33.3 percent for the strongest vision-memory baseline on the button-counting task specifically, where the ambiguity problem is worst, 72.2 percent for FM-VLA there. Strip out the short-state window and force-only performance drops well below the combined system, so force alone isn't the answer either. The two channels are doing different jobs.

The field has defaulted to one memory channel for every kind of temporal problem. This is a clean data point that the channel should match the ambiguity you're actually trying to resolve, not just get bigger.

Source: qft-333.github.io/FM-VLA-Page/
Paper: arxiv.org/abs/2607.18231

Credit to the teams at @Tsinghua_Uni, @MSFTResearch, and @FudanUniversity.

#Robotics #PhysicalAI #RobotLearning

View quoted post on X

Junyi Zhang @ CoRL Verified account @junyi42 · Jun 19

Children learn from play. Can robots do the same?

We propose 𝐏𝐥𝐚𝐲𝐟𝐮𝐥 𝐀𝐠𝐞𝐧𝐭𝐢𝐜 𝐑𝐨𝐛𝐨𝐭 𝐋𝐞𝐚𝐫𝐧𝐢𝐧𝐠, a paradigm that gives embodied coding agents a play stage before downstream tasks arrive, and instantiate it with 𝐑𝐀𝐓𝐬 (Robotics Agent Teams), where robots discover reusable skills through curious play.

Co-led with @jiaxin_ge_

View quoted post on X