MLflow

MLflow 3.18.0 is now available! 🎉

🔍 Automatic Issue Detection finds recurring agent failures across your traces, from wrong answers to unusual spending.
📚 The experimental Agent Skill Registry shares reusable skills with coding agents like Claude Code, Codex, and Cursor.
📊 Evaluation comparisons now pin a baseline and surface regressions.
✅ Experimental online evaluation runs your Python scorers on new traces on a self-hosted server.

Full release: mlflow.org/releases/3.18.…

Five coding agents usually means five API keys and one invoice nobody can explain.

@2twitme shows how to govern that with MLflow AI Gateway. Agents point at one gateway, and it runs guardrails, enforces a budget, and traces every call.

👉 Learn more: mlflow.org/blog/govern-co…

#MLflow #GenAI #LLMOps #OpenSource

MLflow Ambassador @shrinathsuresh covers D for Dataset Tracking in this MLflow A-Z clip.

Log training and evaluation datasets, version them, and associate those versions with your experiments. Skipping that is a huge mistake for GenAI, because the models are probabilistic.

🎥 youtube.com/watch?v=BoTbLW…

MLflow AI Gateway has budget policies. You set a daily, weekly, or monthly spend limit. 👇

When it hits, you can send a webhook to Slack, PagerDuty, or any HTTP alerting and keep serving traffic, or block requests until the window resets. The limit applies to every app that routes through the gateway.

Get started: pip install 'mlflow[genai]'

🔗 mlflow.org/blog/gateway-b…

This week at ODSC AI West, Debu Sinha (@databricks) is running a workshop on Evaluating LLM Applications with MLflow.

Teams already run evals. The workshop is the part after: did the model actually get better, which samples regressed, and is the delta statistically significant or just noise. You build that pipeline in open source MLflow, from mlflow.genai.evaluate() through run comparison and a CI gate that fails the build when a regression is real.

🗓️ Wed, Oct 28
🕓 11:00 AM–12:00 PM PT
📍 Hyatt Regency SFO

🔗 odsc.ai/speakers-portf…