Pinned post
Autoregressive sequence models like transformers develop internal representations of semantically meaningful abstract actions!
Enters Internal RL: An AI inside an AI that solves hard problems by manipulating these abstractions to steer the outer model.
Enters Internal RL: An AI inside an AI that solves hard problems by manipulating these abstractions to steer the outer model.
Standard reinforcement learning in raw tokens is a disaster for sparse rewards!
Here, we propose 𝗜𝗻𝘁𝗲𝗿𝗻𝗮𝗹 𝗥𝗟: acting on abstract actions emerging in the residual stream representation.
A paradigm shift in using pretrained models to solve hard, long-horizon tasks! 🧵View quoted post on X
