Giovanni Monea

@giomonea

🤖 ML PhD student @cornell @cornell_tech

giovannimonea.com Joined July 2023

387 Following 593 Followers 122 Posts

Latest posts 5

Pinned post

Giovanni Monea @giomonea Jul 13

Transformers use the same hidden state for two competing roles: maintaining state and predicting the next token. 🔀

Splitting these roles into two separate streams outperforms standard Transformers.

Introducing the State-Prediction Separation (SPS) Transformer.

🧵

Giovanni Monea @giomonea Nov 2

Join us!

Yoav Artzi @yoavartzi Nov 2

Cornell (and me) is hiring postdocs, with focus on technical AI/ML research. Deadline for full consideration is Dec 15.

I am recruiting through this call as well. I encourage applying as early as possible, because I am starting to review folders now!

Application link ⬇️

View quoted post on X

Giovanni Monea @giomonea Oct 16

Yet another banger paper from Nathan :)

It's been a lot of fun hearing day after day about how powerful his new Copy-Paste Language Models are despite their simplicity, and how effective they are at handling long contexts

Nathan Godey @COLM 🌉 @nthngdy Oct 16

⌘C ⌘V: Introducing Copy-Paste Language Models!

CPLMs learn to copy tokens better, improving downstream accuracy, CoT, long-context performance, and memory-efficiency for RAG at <3% overhead

PS: they (partially) mitigate the LM head bottleneck

Paper: arxiv.org/abs/2610.19247

View quoted post on X

Giovanni Monea @giomonea Oct 6

⚠️ Stop pretraining your looped Transformer with a separate KV cache per recursion!

In our recent preprint, we show that *memory sharing* not only saves memory but is also a net-positive inductive bias!

Less memory, same flops, higher quality.

🔗 arxiv.org/abs/2610.02383

🧵

Giovanni Monea @giomonea Apr 9

LLMs waste massive memory remembering every reasoning step. What if they could leave behind just "breadcrumbs" instead?

Breadcrumbs Reasoning: KV cache compression during decoding with learned beacon tokens. 2–32x less memory, minimal accuracy drop.

🧵