Physics-accurate video generation is a crucial problem to solve if we want to arrive at world models that are actually useful. The key issue is that typical diffusion-based video models don't replicate real-world physics accurately because they simply don't have the right inductive biases to "fit" the exact underlying dynamics of the scene.
We at @DiracRobotics took this challenge head on, and here's our v0 of WorldMatter, a physics-accurate gaussian simulator!
Given a few training samples of a particular type of physical interaction (eg: bowling ball hitting some pins), it's able to learn, and generalize the dynamics to give us accurate rollouts of the scene.
Check out our results in this video. There are definitely still some imperfections, but our approach is getting iteratively better with each experiment we execute.
Stay tuned for our upcoming releases!
We at @DiracRobotics took this challenge head on, and here's our v0 of WorldMatter, a physics-accurate gaussian simulator!
Given a few training samples of a particular type of physical interaction (eg: bowling ball hitting some pins), it's able to learn, and generalize the dynamics to give us accurate rollouts of the scene.
Check out our results in this video. There are definitely still some imperfections, but our approach is getting iteratively better with each experiment we execute.
Stay tuned for our upcoming releases!
The first step towards building a world where robots can learn.
Meet WorldMatter v0 from Dirac.
Our 3D Gaussian world model rolls out world interactions after seeing a few initial frames.
The same rollout can be observed from novel viewpoints, letting us see the interaction from different angles while preserving 3D geometry and plausible physics.
It can generalise across different motions of the world, including complex multi-collision environments.
This is just the beginning.View quoted post on X
