Long Le ✈️ CoRL

omniguide.github.io OmniGuide A composable guidance framework that improves VLA performance by leveraging arbitrary sources of guidance in 3D space.

Kostas Daniilidis Verified account @KostasPenn · Mar 13

VLA policies learn generalist robot behaviors from massive teleoperation datasets, hoping that the right behavior emerges. But they rarely use perception during training or inference: powerful foundation models of 3D geometry, semantics, or human motion are ignored.

@TimSong52005757, @LongLeRobot, and our @GRASPlab team introduce Omniguide, based on a simple idea: Instead of retraining policies, let perception guide action at inference time.

We express diverse guidance sources as attractive and repulsive energy fields in 3D space and inject their gradients into the generative process of VLA policies.

This lets perception modules steer actions without retraining the policy: A modular path toward composable robot intelligence.
Project: omniguide.github.io
Paper: arxiv.org/abs/2603.10052

View quoted post on X

TimSong✈️CoRL Verified account @TimSong52005757 · 6d

VLAs like π0.5 and GR00T still struggle: collisions in clutter, complex semantics, learning from human demos.
OmniGuide steers any flow/diffusion VLA at test time with VLM agents, geometry foundation models & human teaching.
No extra data, no retraining, composable.
#CoRL2026

View quoted post on X

Kostas Daniilidis Verified account @KostasPenn · Mar 13

VLA policies learn generalist robot behaviors from massive teleoperation datasets, hoping that the right behavior emerges. But they rarely use perception during training or inference: powerful foundation models of 3D geometry, semantics, or human motion are ignored.

@TimSong52005757, @LongLeRobot, and our @GRASPlab team introduce Omniguide, based on a simple idea: Instead of retraining policies, let perception guide action at inference time.

We express diverse guidance sources as attractive and repulsive energy fields in 3D space and inject their gradients into the generative process of VLA policies.

This lets perception modules steer actions without retraining the policy: A modular path toward composable robot intelligence.
Project: omniguide.github.io
Paper: arxiv.org/abs/2603.10052

View quoted post on X