Francis Y. Yan

Our second work at ICLR '26 is:

🏟️ 𝙉𝙚𝙩𝘼𝙧𝙚𝙣𝙖: Dynamic Benchmarks for AI Agents in Network Automation. netarena.ai

It was recently recognized as the 🏆 Winner of the AgentX - AgentBeats Competition (Coding Agent Track).

🔎 Key distinction of NetArena: Users can dynamically generate unlimited benchmark tests. This dynamic generation of queries reduces the risk of data contamination and enhances statistical reliability of evaluations.

NetArena currently consists of three network automation tasks: capacity planning, routing misconfiguration, and microservice policy deployment.

🤝 This work was led by @LesleyChou339, a PhD student at UMD advised by @Alan_Lau. Lesley is truly an exceptional collaborator — I've worked with her on 3 projects so far!

The second author is my master's student at UIUC @siebelschool, Jiajun Ruan. This work also involves significant collaboration with my former colleague at @MSFTResearch, Kevin Hsieh, with additional contributions from Sadjad Fouladi and Eric Wang.

Hope it's not too late to share that we had two papers at ICLR '26 on agentic AI.

💡𝙈𝙚𝙩𝙖𝙈𝙪𝙨𝙚: a creative ideation framework for LLM-based algorithm discovery.

As progress continues in improving LLMs' reasoning skills, their creativity 🎨 is lagging behind. We observe that when prompted to generate novel, higher-performing algorithms, LLMs tend to stick to familiar, well-known designs due to inherent biases.

MetaMuse tackles this by steering ideation in a measurable feedback space (i.e., algorithm performance over traces), using external stimuli (e.g., keywords) to diversify thinking and waypoint reasoning to introduce structured checkpoints.

We tested MetaMuse on cache replacement and bin packing, and observed more diverse and higher-quality solutions in both domains.

- HTML slides: illinois-nsai.github.io/MetaMuse/
- Paper: illinois-nsai.github.io/MetaMuse/paper…
- Code: github.com/illinois-nsai/…

🤝 Work started at @MSFTResearch with former colleagues Mike Liang and Yanjie Gao, and intern Ruiying Ma (now a PhD student at Berkeley👩‍🎓). We also thank Encheng Xie (undergrad @siebelschool at the time) for his contributions to the work.

🥁NSDI Mentorship Program🪘If you are attending NSDI '26 in person, you are strongly encouraged to sign up for the Mentorship Program by April 21, either as a mentor or mentee: usenix.org/conference/nsd…

NetArena received 1st place in the "Coding Agents" category at the AgentX-AgentBeats competition (hosted by Berkeley RDI)! Kudos to @choulesley979 and the team.

Code: github.com/Froot-NetSys/N…
Paper (to appear at ICLR '26): arxiv.org/pdf/2506.03231

Join us this Monday at SOSP in Seoul for a day of insights and discussions at the intersection of ML + systems!

💡 The 4th Workshop on Practical Adoption Challenges of ML for Systems (PACMI ’25)
🗓️ Agenda: sites.google.com/view/pacmi/sch…