Twitxy Saved

Jeremy Cohen

@deepcohen

Research fellow at Flatiron Institute, working on understanding optimization in deep learning. Previously: PhD in machine learning at Carnegie Mellon.
New York, NY jmcohen.github.io Joined September 2011
1K Following 6.5K Followers 1.2K Posts

Latest posts 5

Jeremy Cohen @deepcohen
Did Anthropic get more gains out of model scaling than other labs thought was possible? It reminds me of an interesting recent paper, which showed that deep layers in open LLMs are not doing much, and that this can be fixed by scaling the LayerNorm output.
arxiv.org/abs/2502.05795
arxiv.org The Curse of Depth in Large Language Models In this paper, we introduce the Curse of Depth, a concept that highlights, explains, and addresses the recent observation in modern Large Language Models (LLMs) where nearly half of the layers are...
9 replies 22 reposts 298 likes 26K views View on X
Jeremy Cohen @deepcohen
The recent Microsoft AI report noted that too much learning rate decay during pretraining hurts post-RL performance. This is actually just the latest of several papers this year pointing out that small learning rates can be harmful in LLM pretraining. (Thread)
5 replies 22 reposts 197 likes 19K views View on X
Jeremy Cohen @deepcohen
Isn’t it a little ironic that this argument for why LLMs aren’t truly intelligent is based on … pattern matching?
Big Brain AI Verified account @realBigBrainAI Feb 8
AMI Labs founder Yann LeCun on why LLMs are fooling us the same way AI has for decades:

He argues that every generation of AI scientists has made the same mistake: confusing task performance with real intelligence.

LeCun's core challenge to the current hype:

"We're fooled into thinking those machines are intelligent because they can manipulate language. And we're used to the fact that people who can manipulate language very well are implicitly smart."

He's clear that LLMs are useful, but being a useful tool and being intelligent are two very different things.

The real insight is the historical pattern he's lived through.

Since the 1950s, wave after wave of AI researchers have claimed their breakthrough was the path to human-level intelligence.

Marvin Minsky. Herbert Simon. Frank Rosenblatt — who invented the perceptron, the first learning machine, in the 1950s — all predicted machines as smart as humans within a decade.

"They were all wrong."

LeCun has personally witnessed three of these cycles of hype and disappointment. And his verdict on the current one is blunt:

"This generation with LLMs is also wrong. It's just another example of being fooled."

The pattern: A new technique emerges → machines get good at specific tasks → we assume general intelligence

The question worth asking: are we impressed by these tools because they're intelligent, or because they sound like they are?
View quoted post on X