Twitxy Saved

Luo Mai

@luo_mai

Associate Professor @EdinburghUni. Machine Learning Systems. luomai.github.io
United Kingdom luomai.github.io Joined February 2013
414 Following 327 Followers 98 Posts

Latest posts 5

Luo Mai @luo_mai
We will continue to evolve this benchmark and expand its coverage to accommodate new AI accelerator vendors as they emerge.
Edoardo Ponti @PontiEdoardo Sep 1
Developing next-generation accelerators requires benchmarking them on realistic and agentic AI workloads.

With @ARIA_research's Scaling Compute programme, we are releasing TEAS 🍵, serving 5 models (4B to ~1T total) on 9 accelerators for 6 workloads.

teasbench.com
View quoted post on X
Luo Mai @luo_mai
Excited to share a super surprising result from a large collaboration: by coordinating multiple large and small agents, we can improve accuracy by up to 44% over a single-agent baseline, while reducing cost by up to 12X. We started this benchmarking in December 2025, and it has become even more timely with the recent booming discussion on multi-agent vs. single-agent.
Luo Mai @luo_mai
It is my great honor to be invited to give a keynote at this workshop to discuss "What Primitives Get Us to 1000× Serverless AI Inference?" See you tomorrow.
WoAIS 2026 @wosc17 Jun 21
Join us on June 23 for the First International Workshop on AI and Serverless #woais1 #wosc 9 talks on #serverless and AI topics such as AI inference, agents, scaling, profiling, RAG, edge, workflows, and more. See the full program and register for free at serverlesscomputing.org/woais1/
View quoted post on X
Luo Mai @luo_mai
Today I noticed a new repo, “leoheuler/flashtensors”, created only weeks ago, claims to be “built from the ground up” with the exact same speedup as the ServerlessLLM (sllm) model loader. After checking the code and commits, I found several concerning facts:

1️⃣ File names and code structures are identical to ServerlessLLM’s loader (see screenshot below).
2️⃣ Most commits just replace “sllm” with “flashtensors”
3️⃣ Our Apache 2.0 license was removed, and files copied from sllm have no attribution pointing back.
4️⃣ The performance claims come entirely from our loader, with only minor wrappers added.

We’ve archived all evidence here: github.com/future-xy/flas…. We have to do it, since GenAI has made the so-called "refactoring" extremely easy.

Copying and rebranding an open-source project now takes less than a week. The real question: how do we protect the credit for genuine open-source contributors?

ServerlessLLM was built by 20+ contributors over three years. Seeing that work suddenly attributed to an anonymous account on GitHub is disappointing.

Since open-sourcing ServerlessLLM, we’ve been supported and reposted by amazing people like @tom_doerr, @hei_nyan, and @bessi_ludovico, and many have built great systems on top of it (BlitzScale OSDI 2025, Aegaeon SOSP 2025, ...). That’s the real spirit of open source. Copy-and-rebrand? That only undermines the community we’re all trying to build.
Luo Mai @luo_mai
Thanks @tianyin_xu for reposting this on the prestigious SIGOPS blog! We look forward to work with the community on new operating systems research to fulfil the full promise of emerging AI hardware systems.
Tianyin Xu Verified account @tianyin_xu Oct 25, 2025
This's a brilliant article written by @luo_mai's team @EdinburghUni and @fanyang's team @MSFTResearch. It's a must to read for anyone interested in future AI hardware.

Wafer-scale chips are exciting. Today's AI software stack falls short in exploiting them to realize wafer-scale systems.
View quoted post on X