Infini-AI-Lab

New AI data centers are now a fight over power and memory. The cheapest capacity: more from the GPUs we have.

We asked coding agents to build LLM servers from scratch. The best served up to 4.9× the value per GPU dollar of vLLM/SGLang.

ServerBench 🧵
arxiv.org/abs/2611.06841

AI is hitting two walls at once: power and memory.

Voters just pushed back on new data centers over power bills, and HBM and DRAM are in short supply.

Our question: can coding agents make the models themselves smaller?

Introducing CompressionBench 🧵
arxiv.org/abs/2611.06858

Hermit v0.2.0 is out. 🐚🦀

1. Harness evolving with your coding plan: 𝐧𝐨 𝐀𝐏𝐈 𝐤𝐞𝐲 𝐧𝐞𝐞𝐝𝐞𝐝.

2. Now supporting 𝟏𝟐 𝐡𝐚𝐫𝐧𝐞𝐬𝐬 𝐞𝐯𝐨𝐥𝐮𝐭𝐢𝐨𝐧 𝐦𝐞𝐭𝐡𝐨𝐝𝐬: Prompt-only, GEPA, SkillOpt, MWM, Harness², Continual Harness, ReHarness, Meta-Harness, ACE, AHE, RRSI, EvoSafeHarness. (More to come!)

3. A new agent abstraction that makes adding methods much easier.

𝐆𝐢𝐭𝐇𝐮𝐛: github.com/Infini-AI-Lab/…

𝐎𝐫𝐢𝐠𝐢𝐧𝐚𝐥 𝐑𝐞𝐥𝐞𝐚𝐬𝐞 𝐀𝐫𝐭𝐢𝐜𝐥𝐞: x.com/i/article/2114…

We are excited to introduce 𝐇𝐞𝐫𝐦𝐢𝐭 🐚: an open-source framework for training, evaluating, and deploying harness adapters.

Harness evolution is a key building block for RSI and continual learning, enabling agents to continually improve how they work from experience.

Train a reusable external layer. Use existing evolution methods or build your own through one standard framework.

GitHub: github.com/Infini-AI-Lab/…

1/5

#HarnessEvolution #ContinualLearning #RSI #Agent

🧠 How can we keep scaling LLMs with minimal changes in their serving energy?

DeepSeek’s Engram and our concurrent work STEM showed the promise of static sparse memory: large embedding tables with efficient access. But can mature pretrained models learn to use a newly added memory within a limited training budget, avoiding costly pretraining from scratch?

🚀 Introducing STAPLE: a recipe to extend pretrained models' abilities with static sparse memory. Serving energy per token stays close to dense and >50% below MoE upcycling in high-throughput regime.

Scalable capacity, interpretable memory, and compatibility with MoE upcycling - without any meaningful increase in serving cost. 🧵

Paper: arxiv.org/abs/2610.12599
Code: github.com/Infini-AI-Lab/…
Website: infini-ai-lab.github.io/STAPLE/