How much of vLLM’s speedup is actually the scheduler? @nhu_hoang isolated it: continuous batching delivered up to 2.91× over static batching, but exactly 1.00× when every reply had the same length. It all comes down to how much your output lengths vary. towardsdatascience.com/continuous-bat…
Towards Data Science
Verified account This account is verified. Learn more@TDataScience
The world's leading publication for data science and artificial intelligence professionals.
Submit an Article ✍️ contributor.insightmediagroup.io
Submit an Article ✍️ contributor.insightmediagroup.io
1.9K Following 252K Followers 65K Posts
Latest posts 5
A hundred people ask the same AI for ideas. Each gets a better answer than they’d have written alone, and the room ends up with nearly the same answer a hundred times. @DattijoMuhammad weighs the evidence on AI and creativity, and suggests a simple fix. towardsdatascience.com/ask-ai-to-refi…
Two maps found 1,515 forests. A fisheries formula from the 1890s said about 207 were missing from both. Maksym Bilychenko applies the same trick to AI agents, estimating what your eval never sees, with only a 10% hand audit. towardsdatascience.com/how-to-estimat…
OLS predicted $164K for a house that sold for $219K. Quantile regression gave a range, $136K to $227K, that contained it. @gurezende shows how to build prediction intervals that widen where your data gets noisier. towardsdatascience.com/quantile-regre…
Most multi-agent systems come down to two moves: call a specialist agent as a tool, or hand off control entirely. Shuai Guo builds both in Python with the OpenAI Agents SDK, then shows how fan-out, pipelines, hierarchies, and swarms reuse them. towardsdatascience.com/how-to-build-m…