Pinned post
Thrilled to introduce Mirror, our first "learning-from-experience" project! Assume you had a video generative model that, no matter your situation, could always generate video of what it would look like to solve a task. How can you put that plan into action? Clearly, you need to learn some video-to-action mapping - but where do you get the data for that? It turns out you can collect that data in an amazingly sample-efficient way by *incrementally following the video planner*! (1/n)
If video models scale like LLMs, they'll soon generate a perfect video plan for any embodied task. Do we need anything else? Introducing Mirror, which learns to follow these plans just by practicing, and even crafts an iron pickaxe in Minecraft w/o rewards or action labels! (1/n)View quoted post on X
