Pinned post
Jack Morris
Verified account Engram@jxmnop
research @engramlab // language models, information theory, science of AI
1.1K Following 54K Followers 5.9K Posts
Latest posts 5
i'm very excited about this general research, using machine learning techniques to discover the Primordial Essence of things; in this case, it's sea anemones
these are the most Anemone anenomes you'll ever find, even in latent space
someone needs to do this in text 🔜
these are the most Anemone anenomes you'll ever find, even in latent space
someone needs to do this in text 🔜
Can use just 10 images to teach a model what a sea anemone looks like?View quoted post on X
In our #neurips2026 paper, "Dataset Distillation via Drifting," we distill ImageNet's 1,300 images per class down to just 10, optimized from noise.
drifting-distillation.github.io
More info in thread! ⬇️
1/N
these days i'm quite excited about the prospect of models that understand more about our data than we do
one barrier we've run into is that when models run for a long time, they take increasingly inscrutable notes; it turns out that small amounts of training can fix this
also fun to share a glimpse into the kind of work we've been up to at Engram! expect more soon
one barrier we've run into is that when models run for a long time, they take increasingly inscrutable notes; it turns out that small amounts of training can fix this
also fun to share a glimpse into the kind of work we've been up to at Engram! expect more soon
In our new blog, we share some new science investigating how RL complements text-based memory by letting models learn how to use their notes.View quoted post on X
It turns out that training can be cheap compared to building notes, and stays surprisingly robust even through millions of tokens of edits.
Yes, it's true. After years working as a researcher in the field of AI, my jumpshot is finally on camera. And @oneill_c beat me in a friendly game of HORSE. or did he?....
(also thanks to BaseTen for letting me keep the ball, along with many bags of Cheez-Its)
(also thanks to BaseTen for letting me keep the ball, along with many bags of Cheez-Its)
AI researchers' idea of a good time: playing HORSE and debating continual learning between shots.View quoted post on X
@oneill_c (co-head of training at Baseten) and @jxmnop (co-founder and head of research at @EngramLab) discuss:
- when to put knowledge in weights vs. the KV cache,
- why forgetting matters less for specialized models,
- on-policy self-distillation vs. RL, and
- what research publishing should look like today.
Watch the full game (and discussion) here: youtu.be/q_-_t5AmKz0