Mor Geva

This account is verified. Learn more

@megamor2

Assistant Professor at @TelAvivUni and Research Scientist at @Irregular; previously at @GoogleResearch, @GoogleDeepMind and @allen_ai

mega002.github.io Joined April 2017

591 Following 3.1K Followers 654 Posts

Latest posts 5

Mor Geva @megamor2 2d

Fazl took the ambitious goal of surveying the state of interpretability for AI safety. The final result is a valuable set of practical resources for anyone interested in getting into this field. Check it out!

Fazl Barez Verified account @FazlBarez 3d

New paper❗

Does understanding how AI models work actually make them safer?

This assumption often underlines interpretability research, but different subfields operationalise this intuition in very different ways. Our survey of mechanistic interpretability (MI) for safety provides an opinionated guide to the field of MI, detailing conceptual approaches and techniques under this broad banner and how different types of work relates to the mission of making AI model safer. 🧵

View quoted post on X

Mor Geva @megamor2 Oct 29

Now there's a video:
x.com/TiroshDor/stat…

Mor Geva Verified account @megamor2 Oct 1

New preprint: 🔥 Latent Information Feedback Transformers (LIFT) 🔥

Information in LM generation propagates downward (high to low layers) only through the decoded token, creating a bottleneck. Removing it via state propagation makes the model recurrent, which is not scalable for training.

Can we teach LMs to propagate state while keeping pretraining fully parallel? YES!

LIFT is a Transformer-based architecture + teacher-supervised training approach that enables Transformer LMs to exploit deep-to-shallow feedback, while keeping training parallel.

LIFT models consistently outperform standard Transformers and baselines on language modeling, downstream reasoning tasks, and procedural tasks under a token-matched budget, while being on par with or ahead of compute-matched Transformers. LIFT also outperforms Transformers trained on 8x more data on a state-tracking task, even when trained with the states of a Transformer that fails it!

Work by the amazing @TiroshDor and @AmosaurusRex

Paper: arxiv.org/abs/2609.38149
Detailed post + visualizations + code + models coming soon!

View quoted post on X

Mor Geva @megamor2 Oct 29

Check out Dor's fantastic work on teaching Transformers to propagate state in parallel pretraining 👇

Dor Tirosh Verified account @TiroshDor Oct 29

What if a Transformer could feed back more than one token?

In my first paper, we introduce LIFT: an LM that passes a latent state from its last layer back into its first layer at every step, and still pretrains fully in parallel.

w/ @AmosaurusRex @megamor2 🧵

View quoted post on X

Mor Geva @megamor2 Oct 24

Here for 40 hours! 🇭🇺 #EMNLP2026

Will be joining the plenary panel tomorrow, then presenting work with @dhgottesman and @ClaraSuslik on understanding and erasing knowledge representations in LLMs. Come say hi!

dhgottesman.github.io/LMEnt/
arxiv.org/abs/2606.03695

Mor Geva @megamor2 Oct 5

I won't be at #COLM2026, but if you're there and interested in interpretability and knowledge in LLMs, check out papers from our group and collaborators!

📌 Thinking to Recall: How Reasoning Unlocks Parametric Knowledge in LLMs
arxiv.org/abs/2603.09906
Poster Session 1, Oct 6 11:00 AM, Imperial Ballroom, Poster #28
@zorikgekhman @jonherzig

📌 Disentangling MLP Neuron Weights in Vocabulary Space
arxiv.org/abs/2604.06005
Poster Session 4, Oct 7 4:30 PM, Franciscan B, Poster #86
@AsafAvrahamy @GurYoav

📌 Beyond the Rosetta Stone: Unification Forces in Generalization Dynamics, Spotlight!
arxiv.org/abs/2508.11017
Oral Session 4, Oct 7 3:30 PM,
Poster Session 4, Oct 7 4:30 PM, Imperial Ballroom, Poster #4

Also consider attending the (2nd) Actionable Interpretability Workshop -- we have an exciting lineup of speakers and panelists!
actionable-interpretability.github.io
@ActInterp

arxiv.org Thinking to Recall: How Reasoning Unlocks Parametric Knowledge in LLMs While reasoning in LLMs plays a natural role in math, code generation, and multi-hop factual questions, its effect on simple, single-hop factual questions remains unclear. Such questions do not...