Twitxy Saved

Sanae Lotfi

Verified account This account is verified. Learn more

@LotfiSanae

AI Research Scientist @MetaAI (FAIR) | PhD from @nyuniversity
Menlo Park, CA sanaelotfi.github.io Joined August 2020
445 Following 2.9K Followers 483 Posts

Latest posts 5

Sanae Lotfi Verified account @LotfiSanae
What if we thought of layers as flexible building blocks that can be rearranged and parsed in an optimal order conditioned on the input itself? Our new work explores this question👇

Big congrats to @arahmani_AR, @SeanMcleish, @micahgoldblum on this great work!
Sean McLeish Verified account @SeanMcleish Nov 3
Library of Layers: models which adapt layer order per input, routing over whole transformer blocks.

This opens up a new avenue to control: the execution path the model uses. We use this to control the algorithm used to solve mazes and sudoku by only changing the layer order.
🧵 1/n
View quoted post on X
0 replies 3 reposts 21 likes 1.1K views View on X
Sanae Lotfi Verified account @LotfiSanae
🚨New paper: Small-Scale Experiments: Are We There Yet?

Scaling laws promise that we can run cheap experiments on small models and extrapolate to large ones. But in practice, fits from small models have often looked unreliable, leaving practitioners convinced that experiments on fairly big models are unavoidable.

We find that small models need much more hyperparameter tuning than larger ones. With enough tuning, we get scaling laws down to 4M parameters, which train in under an hour on one GPU.

We might be the last people to post about our own work, but we took time to digest the feedback from the community and put together a blog that I hope is easier to follow: reliable-ai.review/posts/2026-10-…
4 replies 15 reposts 100 likes 5.5K views View on X
Sanae Lotfi Verified account @LotfiSanae
Our work was selected for a #NeurIPS2026 Oral (top 0.3%)!! 🎉

I'm glad the PC recognized the importance of token efficiency as a research direction beyond quantization, and the simplicity of what we propose here.

We burn billions of dollars on tokens and a lot of these tokens are imo unnecessary filler tokens, verbose prose, or poor planning that you simply can't address with a length penalty.

While our work here is more focused on inference-time interventions, getting LLMs to communicate directly, concisely, and to stick to the necessary level of reasoning should be addressed much earlier in the training. Longer reasoning can, and in many cases does, result in poorer performance.

Paper: arxiv.org/abs/2606.00206
With amazing collaborators @polkirichenko, Steven Li, and Zechun Liu
Sanae Lotfi Verified account @LotfiSanae Jun 25
When a quantized reasoning model fails, we usually assume it just lost the capability to solve the problem. But we surprisingly show that in up to 52% of failures, the quantized model actually reaches the correct answer but then does not commit to it. 🧵👇 1/8
View quoted post on X
29 replies 33 reposts 600 likes 48K views View on X
Sanae Lotfi Verified account @LotfiSanae
Muse Glimmer is out, and we're opening the weights! Genuinely proud of what we shipped here, and even prouder of the people I got to work with on this model. Lots still to figure out, which is the fun part 🎉

More here: research.meta.ai/blog/introduci…
research.meta.ai Introducing Muse Glimmer: An Open Agentic Model That Runs on Your Device Muse Glimmer is a 30-billion-parameter open agentic model from Meta Superintelligence Labs, optimized for always-on local workflows on consumer hardware.
Mark Zuckerberg Verified account @finkd Aug 10
Today we're also opening the weights for Muse Glimmer, a great 30B parameter dense model that can run locally. Soon we'll also release the weights for Muse Spark 1.2, our latest foundation model. Meta is a strong supporter of open source and I'm proud of these releases. Congrats to @alexandr_wang and the MSL team for all your great work on these models.
View quoted post on X
8 replies 2 reposts 32 likes 2.8K views View on X
Sanae Lotfi Verified account @LotfiSanae
Very nice to see our recently proposed overthinking penalty tested and working for others, leading to ~30% cot length reduction here! As we mention in the paper, the penalty on overthinking markers improves efficiency for both quantized and full precision models without hurting the performance.

Our work for reference:
0xSero Verified account @0xSero Jul 5
I found a way to make REAP so much better!!!!

Basically add a penalty to words that lead to self doubt it’s already working!!!

Reduce total tokens output by 30%~

Also experimenting with improved tok/s on CPU offloading
View quoted post on X