Twitxy Saved

Manu Gaur

Verified account This account is verified. Learn more

@gaur_manu

used to do physics, now working on long horizon robotics @CMU_Robotics
New Delhi, India manugaurdl.github.io Joined May 2012
977 Following 1K Followers 2.5K Posts

Latest posts 5

Pinned post

Manu Gaur Verified account @gaur_manu
Pretrained ViTs like DINOv2 or CLIP are great, but they produce fixed, generic representations that encode the most salient visual concepts (e.g., "cat").
In human vision, prior priming with language changes how people parse an image. We believe visual encoders should do the same
🚨 Introducing Steerable Visual Representations, a new family of visual features you can steer with text towards specific visual concepts.
13 replies 136 reposts 918 likes 162K views View on X
Manu Gaur Verified account @gaur_manu
what a beautiful graphic
Wenhao Chai Verified account @wenhaocha1 14h
LLM pretraining has run on single-epoch text, bits per byte and iso-FLOP curves for years. visual generation still memorizes ImageNet for tens to hundreds of epochs and pushes FID down with CFG, with no strong signal those tricks transfer to real text-to-image.

new blog: 🎨 scaling generative models on GPIC, 100M permissive text-image pairs.

training pixel-space JiT from 3.6e18 to 2.3e20 FLOPs:
1) held-out loss is our main metric, and compute-optimal TPP sits at about 50 image tokens per image parameter
2) EMA is not necessary: a proper learning-rate schedule beats EMA on held-out loss at every budget by 0.4% to 1.0%
3) we use CFG = 1 (only the conditional branch) for every result, and call on everyone to drop CFG: it saves compute and leaves no room to hack the metric

all code, configs and sweep data are open 🧵
blog: wenhaochai.com/blogs/scaling-…
View quoted post on X
Manu Gaur Verified account @gaur_manu
they solved unpaired translation. this is so cool!!!
Dominik Schnaus @dominik_schnaus Oct 9
DINOv2 has never seen a caption, and Qwen3 has never seen an image. We still aligned their embedding spaces without a single image-caption pair.

It even works when the images and the captions come from different datasets.

Project page: dominik-schnaus.github.io/unpaired-roset…

⬇️
View quoted post on X
2 replies 4 reposts 93 likes 8.8K views View on X
Manu Gaur Verified account @gaur_manu
Existing efforts for visual memory (e.g. keyframes selection) is suboptimal for long-horizon robotics. This is a great step towards latent memory - use VLMs to supervise the latent during training, while preserving fast control during inference!
Nitish Dashora Verified account @DashoraNitish Oct 1
There’s lots of buzz around agentic harnesses for robots, particularly for long horizon tasks that require complex reasoning and memory. But what will it really take to turn a reasoning agent into a reactive, reliable and low-latency robot policy?

In our new paper, workspace models, we design a new memory architecture that acts as a latent harness for stronger reasoning models. Here’s why we think this might be the way forward:

(1/9)
View quoted post on X