Asher Trockman

The distillation debate is looking pretty one-sided around here! I've been thinking about this for quite a while (along with Yash Savani), so let me add some variety. Regardless of where you stand on the IP issues:

Distillation attacks make the AI ecosystem LESS original, less safe, and -- believe it or not -- also less open. We expand on these points in a blog post linked in the replies.

Our favorite realization is that distillation attacks lead to AI monoculture, and this monoculture could create unanticipated, systemic security risks for both individuals and companies.

When LLM demand exceeds capacity, which requests should go first?

Model providers already offer a few priority tiers. But even within a tier, requests compete for compute, and users value latency differently. Your coding agent might be blocked due to a request that belongs to an unattended background job.

Why not let the market decide?

In our new paper, Inference Auctions, users bid for request priority, and our auction allocates that priority to those who value it most.

Unfortunately, naively selling to the highest bidders destroys prefix-cache locality, increasing SGLang's average latency by up to 12x in our experiments.

Our fix is to run an auction over depth-first search traversals of the request prefix tree, ordering subtrees by average bid. This maximizes social welfare while retaining cache optimality, with VCG payments incentivizing truthful bidding, and the resulting request order preserves key performance metrics (cache hits and mean TTFT) of the underlying scheduler.

w/ @keegan_w_harris @sidpra5ad @nhaghtal and Michael Jordan

Paper: arxiv.org/abs/2609.40070
Project page: inferenceauctions.com

Nicholas Roberts?

check out this great work by @YixuanEvenXu and @jwkirchenbauer

where else can you