Twitxy Saved

Matthias Niessner

Verified account This account is verified. Learn more

@MattNiessner

Professor for Visual Computing & Artificial Intelligence @TU_Muenchen
Co-Founder @synthesiaIO
Co-Founder @SpAItial_AI
Munich, Bavaria niessnerlab.org/publications.h… Joined March 2015
336 Following 51K Followers 1.9K Posts

Latest posts 5

Matthias Niessner Verified account @MattNiessner
Making Echo-3 scans much more useful - now with segmentations, individual objects, and much more :)
SpAItial AI Verified account @SpAItial_AI 4d
Understanding a 3D scene starts with object segmentation. This is why we’re launching a Segmentation API for 3D worlds.

Give it a Gaussian splat world and it splits the scene into individual objects. Each object gets a label and can be exported as its own mesh.

This enables better scene understanding and editing, and is the starting point for scene physics.

Coming soon to the SpAItial API. For early access, reach out to partnerships@spaitial.ai
View quoted post on X
4 replies 4 reposts 55 likes 3.8K views View on X
Matthias Niessner Verified account @MattNiessner
~1 year ago: AI coding didn't work
~0.5 year ago: agentic tasks failed
~now: nobody checks code + full automation

What's missing?

My first (real) attempt at AI coding was mid last year, mostly playing around with Copilot. It was still very much a vibe-coding experience and not ready for serious work. People oversold it -- lots of bugs and way too many iterations to be productive. In fairness, I gave up on it quite quickly.

The real game changer was Opus 4.6 -- it needed precise instructions, but its execution was on another level. If you iterated on the plan until it was well-specified, it would turn each task into pretty decent code. However, it still struggled with longer-term reasoning, and with every iteration the chance of breaking something else went up -- agentic tasks in particular mostly failed.

This changed over the past few months: agentic tasks now work quite well, and I'm trusting agents with much more complicated prompts -- if the underlying methods are well known, this works remarkably well. I'm particularly stunned by how well the VLMs inspect results and propose fixes on their own; 3D modeling is a prime example here but there are many other examples.

Despite that progress, there are still missing pieces:
- the main bottleneck seems to be prompting and the imperfect nature of language itself, i.e., I find it quite difficult to tell agents what I actually want them to do.
- ideation / brainstorming still struggles with specific solutions that aren't mainstream. Especially for research projects, agents rarely come up with good ideas and auto-research loops are mostly limited to hyperparameter optimization.

Given the current insane speed of progress, I'm extremely excited about what the next year brings. I've heard so many times now that "AI can't do x" -- and now we all take this for granted. I also really don't see things slowing down, since everything keeps getting easier to try out. Really looking forward to it!
16 replies 9 reposts 109 likes 8K views View on X
Matthias Niessner Verified account @MattNiessner
📢📢 Cutting-edge research in #AI? 📢📢

We have several fully-funded 𝐏𝐡𝐃 & 𝐏𝐨𝐬𝐭𝐃𝐨𝐜 𝐨𝐩𝐞𝐧𝐢𝐧𝐠𝐬 in our Visual Computing & AI Lab in Munich!

Topics have a strong focus on Generative AI, Agentic 3D Understanding, 3DGS, Diffusion, World Models and many more!

👇👇
1:00
7 replies 29 reposts 213 likes 11K views View on X
Matthias Niessner Verified account @MattNiessner
Congrats to @manuel_dahnert for his successful PhD defense! Manuel’s thesis is on 𝐇𝐨𝐥𝐢𝐬𝐭𝐢𝐜 𝟑𝐃 𝐒𝐜𝐞𝐧𝐞 𝐔𝐧𝐝𝐞𝐫𝐬𝐭𝐚𝐧𝐝𝐢𝐧𝐠 𝐟𝐫𝐨𝐦 𝐚 𝐒𝐢𝐧𝐠𝐥𝐞 𝐈𝐦𝐚𝐠𝐞.

With Panoptic 3D Scene Reconstruction (NeurIPS'21), he introduced a new task: jointly predicting complete 3D geometry, semantics, and instances from just one RGB image. His Coherent 3D Scene Diffusion (NeurIPS'24) proposed generative priors to this problem, and his earlier work on Scan2CAD and joint scan-CAD embeddings helped bridge real-world scans and clean CAD models.

Beyond research, Manuel had an enormous impact on teaching. As head TA of Introduction to Deep Learning (I2DL) for 6+ semesters, he likely taught more than 10,000 students and shaped the AI curriculum at TUM.

Manuel was also a cornerstone of our lab's social life: he organized many lab retreats, sports activities, and countless events that made our group so much more than just a workplace.

Manuel is now bringing spatial intelligence to large-scale digital twins as a 3D ML engineer at NavVis. This is a super exciting area and I'm keen to see what he will be doing next!

Check out his work: manuel-dahnert.com

Congrats Dr. Dahnert, it was a pleasure to advise you; your contributions to the lab will be deeply missed!
3 replies 2 reposts 79 likes 7.4K views View on X
Matthias Niessner Verified account @MattNiessner
Today, we're launching Echo-3, our world model for reconstructing real spaces.

Key idea: treat reconstruction as conditional generation. The outputs are photorealistic 3DGS, point clouds, and meshes at metric scale -- only from a short iPhone video.

We believe this is an important step for physical AI, which is bottlenecked by real-world 3D data. When any phone can produce a metric digital twin, capture scales with phones rather than expensive scanning rigs, for robotics as much as for construction and real estate.

Echo-3 is also available via API: photos or video in, 3D worlds out. Check it out!
SpAItial AI Verified account @SpAItial_AI Oct 27
Introducing Echo-3. A state-of-the-art world model.

Echo-3 turns phone captures into photorealistic, metric-scale 3D worlds. From a single room to multi-floor buildings, outdoor and public spaces.

Available today.
View quoted post on X