Twitxy Saved

Dheeraj Varghese

Verified account This account is verified. Learn more

@dhevarghese

PhD candidate @UvA_Amsterdam
Amsterdam, The Netherlands dhevarghese.github.io Joined June 2021
459 Following 71 Followers 30 Posts

Latest posts 5

Dheeraj Varghese Verified account @dhevarghese
Really cool work led by @iWSimoncini! 🚀

The idea is simple but powerful: what if a single pretrained Transformer could adapt to different compute budgets at inference time, without retraining?

One learned ranking of neurons and attention heads makes this possible. Check out the thread! 👇
Walter Simoncini Verified account @iWSimoncini Oct 15
🪗 Your Transformer Is Secretly Elastic 🧵👇

Transformer families come in a few fixed sizes, so you commit to one for your hardware and latency constraints, even as query loads vary. What if a single pre-trained model could serve every budget?

We learn one importance ranking over every MLP neuron and attention head, once, with frozen weights. At inference time, the model can then be pruned to any computational budget by dropping its least important units, without retraining.

Across 27 vision and text encoders spanning 22M to 8B parameters, performance degrades smoothly up to 60% sparsity. For example, pruning DINOv3 ViT-H+/16 to 50% sparsity removes 427M parameters, for only a 0.8% drop in ImageNet-1k linear probing.

Super fun project in collaboration with Dheeraj Varghese (@dhevarghese), supervised by Gertjan J. Burghouts (@gjburghouts) and Cees Snoek (@cgmsnoek)

🌐 Project page: secretly-elastic.ashita.nl
View quoted post on X
Dheeraj Varghese Verified account @dhevarghese
Ever wondered whether AI agents could play thousands of video games on a supercomputer? We did. 🎮

Introducing PlaySuite: 5,734 games, 14 open models, and a large-scale benchmark for interactive visual intelligence.

Here's what we built, and what we learned. 🧵👇
0:20
4 replies 9 reposts 25 likes 3.3K views View on X
Dheeraj Varghese Verified account @dhevarghese
Excited to be a part of this! ✨
Cohere Labs Verified account @Cohere_Labs Oct 28, 2025
3 days. Worldwide. Inspiring & starting new research collaborations. Introducing the Connect conference. 🖇️

Join for incredible speakers, including @1vnzh @jpineau1 @mziizm & @ShayneRedford + >20 researchers discussing how collaboration and open science are driving progress. 🚀
View quoted post on X
Dheeraj Varghese Verified account @dhevarghese
Through NeoBabel, I’ve experienced how large-scale distributed training works in practice. @mmderakhshani's walkthrough distills those lessons, illustrating the five key parallelism strategies. A great read👇
Mohammad Mahdi (Medi) Derakhshani Verified account @mmderakhshani Oct 15, 2025
🔗 Inside 3D Parallelism: Scaling Transformers in PyTorch (A Code-Study Walkthrough):
tinyurl.com/e6kzye4j
View quoted post on X