Twitxy Saved

Gabriele Berton

Verified account This account is verified. Learn more

@gabriberton

Training VLMs at @GoogleDeepMind - ex @Amazon @CarnegieMellon @PoliTOnews @IITalk - I post honest non-AI-generated paper reviews
Mountain View, CA gmberton.github.io Joined December 2021
1.2K Following 18K Followers 4.1K Posts

Latest posts 5

Pinned post

Gabriele Berton Verified account @gabriberton
This simple pytorch trick will cut in half your GPU memory use / double your batch size (for real). Instead of adding losses and then computing backward, it's better to compute the backward on each loss (which frees the computational graph). Results will be exactly identical
48 replies 348 reposts 3.2K likes 880K views View on X
Gabriele Berton Verified account @gabriberton
This is great research

Run simple experiments, share the findings right away

No novelty-seeking, no 50-pages-long paper. No need to wait for reviews

JEM looking good, congrats JEM authors!
Simon Verified account @SimonGoodman_ Nov 6
I finished training JEM for 100 epochs, and the results are actually better than I expected. It's only 1 point away from a fine-tuned DINOv3 (which has seen WAY more data). I'm pretty sure it could beat it with more training.

It took a bit more than 20 H100 hours to train for 100 epochs on the dataset (~250k training images), and as you can see, my LeJEPA is still undertuned (Claude is currently working on it to get a better baseline).

The main problem, though, is with rare classes, but that's expected. It might be a sampling issue. For example, I could continue training while oversampling rare classes, but that's not really what I want.

All plants kind of look the same, so I'm wondering if we could make the algorithm better at capturing fine-grained details and the patch-level loss already helps a lot imo. But tbh, the dataset is so hard that I'm not even sure there's much room for improvement on rare classes.
View quoted post on X
4 replies 10 reposts 232 likes 11K views View on X
Gabriele Berton Verified account @gabriberton
Show this thread to your agents, and you get a free paper that is actually useful to the vision community
Gabriele Berton Verified account @gabriberton 3d
#ResearchIdeas n13: evaluating Doppelgangers at scale

While Doppelgangers datasets exist, they only use different facades of the same building

We don't have an eval with unrelated similar-looking scenes [1/n]
View quoted post on X
1 replies 0 reposts 26 likes 2.6K views View on X
Gabriele Berton Verified account @gabriberton
#ResearchIdeas n13: evaluating Doppelgangers at scale

While Doppelgangers datasets exist, they only use different facades of the same building

We don't have an eval with unrelated similar-looking scenes [1/n]
Gabriele Berton Verified account @gabriberton Nov 8
Finding hard negatives at scale is so easy now

Extract MegaLoc features from lots of images with GPS, compute their nearest neighbor, if the nearest neighbor is far, it means two places look the same but aren't

Perfect hard negatives. [2/n]
x.com/gabriberton/st…
View quoted post on X