Twitxy Saved

Liangming Pan

Verified account This account is verified. Learn more

@PanLiangming

Assistant Professor, Peking University (@PKU1898) | Former AP @UofAInfoSci | Postdoc @ucsbNLP | Ph.D. @NUSingapore | Researcher in NLP, LLMs & Reasoning
Beijing, China liangmingpan.bio Joined December 2017
911 Following 2.1K Followers 156 Posts

Latest posts 5

Liangming Pan Verified account @PanLiangming
🎉 Our paper "Do Latent-CoT Models Think Step-by-Step? A Mechanistic Study on Sequential Reasoning Tasks" is accepted to #NeurIPS2026!

Latent Chain-of-Thought models reason through continuous hidden states rather than explicit text. But do they really reason step-by-step—or can they reach the right answer without preserving the full reasoning path? We open up CODI with mechanistic analysis to study what happens inside its latent reasoning, and how that changes as reasoning chains get longer.

📝 Paper: arxiv.org/abs/2602.00449

🧵 (1/6)
arxiv.org Do Latent-CoT Models Think Step-by-Step? A Mechanistic Study on... Latent Chain-of-Thought aims to enable step-by-step computation without emitting long rationales, yet its internal mechanisms remain unclear. We study CODI, a continuous-thought teacher-student...
2 replies 10 reposts 59 likes 4.5K views View on X
Liangming Pan Verified account @PanLiangming
Thanks Mingyu for the recognition of our paper!
Mingyu_Jin19 Verified account @fnruji316625 May 24
“Mechanistic Data Attribution: Tracing the Training Origins of Interpretable LLM Units”

One of the papers I personally found very promising earlier this year just got accepted as an ICML Oral. huge congrats to the authors 🔥

The core objective of this work is, in essence, to address a fundamental question within the field of mechanistic interpretability: Which specific training data points were responsible for teaching a particular mechanistic circuit, neuron, or attention head? Put another way: Why do induction heads emerge? Which specific training samples drove their emergence? Can the source of training data for a given interpretable unit actually be traced? The authors term this problem: Mechanistic Data Attribution (MDA).

The core contribution of this paper lies in: Working backward—Starting from the "model mechanism". To deduce: "Which specific training data points led to the formation of that mechanism?" (It is like MI+Influence Function)

I find something interesting in this paper, as many seemingly “low-quality” repetitive data samples may actually serve as critical catalysts for the emergence of induction mechanisms in transformers. And there are many other interesting insights in this paper.

Paper: arxiv.org/pdf/2601.21996

Congrats again to the authors — very well deserved Oral recognition @PanLiangming

#ICML #MachineLearning #LLM
View quoted post on X