Pinned post
Vmax
Verified account This account is verified. Learn more@VmaxAI
RL research focused on open-ended learning
1 Following 652 Followers 64 Posts
Latest posts 5
Nethack is a really underrated benchmark
astra playing NetHack:View quoted post on X
- score: 48,978
- BALROG progress: 49.4%
- max dungeon depth: 17
- max experience level: 14
important note: this is a cherry-picked episode (best of 10), and Astra has been able to modify its own game-playing harness
no privileged information or unfair game modifications, but it uses tools such as pathfinding to previously visited landmarks and code-written trackers for previously observed entities. these offload a fair amount of low-level reasoning, while Astra remains responsible for choosing what to do at every step
imo, the ability of LLMs to autonomously create tools that amortize low-level control costs (and then interpret and use their outputs in-context) is remarkable, and one of the reasons i'm excited about LLM agents. like, we could run the same tool computations and... concatenate the outputs into a flat vector for an MLP... but it would still have to learn what those outputs mean and how to use them. with LLMs we get free hierarchies and natural separation of responsibilities...
anyway, i'm now running episodes with the default settings from BALROG for a more fair comparison
the auto-research programme continues. will report back!
full run below 猡碉笍
You should try Silico. Not only for mech interp, but for AI Research in general.
Goodfire have put a lot work into building a great interface for long-horizon agent-driven research.
Goodfire have put a lot work into building a great interface for long-horizon agent-driven research.
Silico, the platform for ambitious AI research, is publicly available today.
AI is advancing fast. The tools to understand it need to advance even faster. Silico lets you interpret and train your models at frontier scale.
Learn more + get access 馃УView quoted post on X
Following the blog post from our collaboration with @GoodfireAI, the arxiv paper for PROPEL is now available.
The arxiv is now live! arxiv.org/abs/2606.18284View quoted post on X
Exciting new research led by @lorenz_wlf to accelerate RL task generation using mech interp methods in collaboration with @GoodfireAI
Training a model to generate RL tasks not too hard, not too easy costs many solver runs per task.
PROPEL predicts difficulty via a probe on its activations instead, amortizing cost and speeding up generator optimization.
New open-ended RL research from @Vmax + @GoodfireAI.View quoted post on X