Marcel Torné ✈️ CoRL 2026

Marcel Torné ✈️ CoRL 2026 @marceltornev · Jul 1

We should stop optimizing robot policies against a single overall reward. Trajectories differ along many axes, such as speed, precision, and subtask completion, and one can be better on some while worse on others. If we collapse all of that into a single overall axis we lose this structure making the reward ambiguous and harder to optimize.

Blog: freeform-pl.github.io/fpl.website/
Paper: arxiv.org/abs/2606.32027

View quoted post on X

arxiv.org Freeform Preference Learning for Robotic Manipulation Reward design remains a central bottleneck for autonomous robot policy improvement, especially in long-horizon manipulation tasks where sparse success labels provide too little signal and binary...

RoboPapers Verified account @RoboPapers · Sep 9

Specifying reward functions for robots is one of the hardest things about reinforcement learning. Robot rewards often need to be very detailed; metrics like progress can be ill-defined and hard to estimate. This leads to most robot learning defaulting to sparse rewards or simple preference learning. But naive preference learning (having human annotators choose one trajectory over another) is an easy solution, but obscures a lot of the signal in complex tasks and can make learning a lot less efficient.

@marceltornev, @anubhamahajan01, and @AbhijnyaBhat join us to talk about their solution: freeform preference learning, which lets annotators define natural-language axes to compare trajectories over. This improves real-world performance on long-horizon manipulation tasks over sparse rewards and simple binary preference learning.

Watch Epsiode 103 of RoboPapers, with @micoolcho and @DJiafei, today to learn more!

View quoted post on X