Xingyu Fu

sear.bot SEAR: Self-Evolving Agentic Robotics SEAR is a framework and benchmark for self-evolving robotics: frontier language agents refine control code, carry skills and memories across tasks, and train action models on their own rollouts.

Bangzheng Li Verified account @BangzhengL · Oct 7

Amazed by Astra controlling robots? It turns out other LLMs can do it too – even the text-only ones like DeepSeek. We also find Astra did best watching and correcting every move live; Fable did best building its own tools and policy in code first.

We introduce SEAR, a new benchmark and framework to rigorously evaluate and study how LLM agents self evolve on robots.

🔁📈The loop: within a controlled time budget, an agent and its library update a policy → the policy runs in simulation or on hardware → a judge returns feedback → the library carries the skills and lessons into the next task. 

💻🚏 Self-evolving agents operate in myriad ways - we study all of them at once: the policy can be code the agent wrote (code-as-policy), the agent itself in the control loop (agent-as-policy), or a VLA it trained (autoresearch).

🧑‍🔬🤖We spent 4+ years of accumulated agent evolving time across simulation and real hardware and tested 7 frontier models (GPT-6-Astra, Fable 5.1, Opus 5, GPT-5.6-sol, Gemini-3.6-flash, Kimi-k3, DeepSeek-v4-flash) on 330 sim tasks, 2 physics engines, and real robots with different embodiments. 

🌐Project overview: sear.bot 

A thread 🧵1/5

View quoted post on X

arxiv.org Reinforced Attention Learning Post-training with Reinforcement Learning (RL) has substantially improved reasoning in Large Language Models (LLMs) via test-time scaling. However, extending this paradigm to Multimodal LLMs...

Bangzheng Li Verified account @BangzhengL · Feb 5

🚨 Tokens are the surface.
Attention is the mechanism.
What if RL learned the latter?

💡 Introducing Reinforced Attention Learning

🧠 Key idea: Current RL for LLMs shapes "what is generated". We instead optimize "where the model focuses", to influence how it actually reasons.

📰 Paper: arxiv.org/pdf/2602.04884
(with @Google and @GoogleDeepMind)

Standard PPO/GRPO perform importance sampling over token distributions, using advantages to up- or down-weight token probabilities.

⏳ The flip: Instead of "what token to generate", we optimize "where to attend"

At each generation step, we measure the divergence between attention distributions of the current and old policy—across all previous tokens.

- High-reward samples → keep attention close
- Low-reward samples → push attention away

The policy learns where to allocate computation, not just which surface-level token to generate.

⚙️ Plug-and-play
Our attention-level objective pairs with any policy gradient method, including PPO and GRPO.

📊 Results
RAL > Vanilla GRPO.
We’re seeing consistent accuracy gains across a wide range of image and video QA benchmarks when experimenting on multimodal LLMs.

✨ One more thing
Attention can be learned from a teacher model via on-policy attention distillation—going beyond knowledge transfer to inherit latent attention behaviors. This On-policy Attention Distillation takes standard On-Policy Distillation to the next level.

Thanks to all my colleagues: @jianmo_ni @Chen_Qu1 @ianmiao @liuyang_irnlp @XingyuFu2 @infolaber🙏

View quoted post on X

Zhuang Liu Verified account @liuzhuang1234 · Dec 9, 2025

Excited to work with new PhD students (Fall 2026) on multimodal models, AI for automated scientific research, and foundation model architectures at Princeton.

If this resonates with you, please apply to the CS PhD program and mention my name.

View quoted post on X