The artifacts released with this fully open looped transformer should be a goldmine for anyone wanting to do some analysis studies on top of it! And impressive eval results! Super work by @yuekun_yao and team!
Looped LLMs are heating up🔥. Open weights are becoming available, but the recipes to train these models remain hard to find.View quoted post on X
Introducing ARLoop, a family of strong, fully-open pretrained looped transformers with Attention Residuals! Our strongest model, ARLoop-1.2B, outperforms both Huginn-3.5B, a prior fully open looped LLM, and Qwen3.5-2B Base on evaluated benchmarks across commonsense reasoning, math, and coding.
We release the full recipe: from data preparation and model training to intermediate checkpoints and evaluation code. ⤵️