Akarsh Kumar

This account is verified. Learn more

@akarshkumar0101

PhD Student @MIT_CSAIL
RS Intern @SakanaAILabs
RL, Meta-Learning, Emergence, Open-Endedness, ALife

akarshkumar.com Joined February 2018

715 Following 2.9K Followers 570 Posts

Latest posts 5

Pinned post

Akarsh Kumar @akarshkumar0101 Jun 7

We never really knew how to train nonlinear RNNs well… BPTT struggled with vanishing grads (no long-range memory) and sequential rollout (hard to parallelizable).

What if instead an oracle told us the optimal memory state m_t at each step? Then the RNN could do one-step supervised learning on (m_t, x_{t+1}) → m_{t+1} labels.

We call this Supervised Memory Training (SMT): a replacement for BPTT that trains RNNs without unrolling them. SMT is time-parallelizable and solves vanishing gradients.

Website: akarshkumar.com/smt/
arXiv: arxiv.org/abs/2606.06479

Akarsh Kumar @akarshkumar0101 Nov 2

!!

The axes along which organisms can adapt are often the axes in which the environment changed in the past

François Chollet Verified account @fchollet Nov 1

Evolution doesn't benchmaxx. It optimizes for generality and robustness, because overspecialization makes a species fragile when the environment inevitably shifts.
View quoted post on X

Akarsh Kumar @akarshkumar0101 Oct 9

Check out this conversation on the field of artificial life and the underappreciated power of evolutionary algorithms!

With all these new autonomous AIs running around, ideas from ALife are going to become more and more important.

Thanks @MLStreetTalk @ecsquendor for having me!

Machine Learning Street Talk Verified account @MLStreetTalk Oct 9

"Evolution is anything but random."

This is INSANE...

@akarshkumar0101 (@MIT_CSAIL, @SakanaAILabs) on artificial life: searching the space of possible universes with foundation models, why evolved networks find structure SGD misses (with @kenneth0stanley), and LLMs evolving Core War programs.

View quoted post on X

Akarsh Kumar @akarshkumar0101 Oct 7

Everyone is worried about what research to do in the age of AGI.

I think this ethology of agents perspective is one of the most fascinating topics right now...

More and more AI research is going to resemble natural science rather than engineering new systems.

Great work by @nacloos and @noranta4 !!

Nathan Cloos Verified account @nacloos Oct 7

Is this machine playing? 🏀🪁🛝

We left Eko, an embodied coding agent, on an island for 30 hours with no task or goal, studying it like ethologists study a new species

Eko explored, built, climbed, made games, and learned completely new techniques (1/9)
machineplaying.org

View quoted post on X

Akarsh Kumar @akarshkumar0101 Oct 2

In this work, we formulate forgetting as a geometric distribution shift on a layer's inputs.
The core insight is to update the function behavior only locally at the new data points.
Since gradient learning can often perform unpredictable global function changes, we gate the update using classical non-gradient techniques to explicitly enforce locality.

Great work led by @abk_tau!

Assaf Ben Kish Verified account @abk_tau Oct 2

(1/7) Humans can acquire many capabilities without (catastrophically) forgetting previous ones. Why can't neural networks do the same?

Local Support Learning (LSL) takes us a step closer towards this goal.
In LLMs of up to 7B parameters, LSL augments gradient-based training to retain prior capabilities while learning new tasks at full capacity. Notably, it does so without access to prior data.

Key idea: casting forgetting as a geometric problem, and optimizing for the worst case.

Paper: arxiv.org/abs/2610.02126
Website and code: assafbk.github.io/lsl

View quoted post on X