Red Hat AI

Everything you need to start self-hosting an open LLM.

Run it on your own hardware. No API keys. No per-token bill. Nothing leaves your machine.

The full path with @vllm_project: batch inference in Python, an OpenAI-compatible API server in one command, and quantized models that cut an 8B from ~16GB of weights to a quarter of that while keeping 98-100% accuracy.

Walkthrough by @cedricclyburn.

76 guest speakers. 24 organizations. 59 sessions and counting.

vLLM Office Hours is a bi-weekly livestream with the people building @vllm_project: a project update, a guest talk, and open Q&A. Open to all.

Get a calendar invite: vllm.ai/office-hours

Getting benchmark numbers is easy. Getting the right ones is hard. KV cache offloading can cut TTFT from 60s to 10s and lift throughput up to 5.8x, but only when your working set and storage tier line up. Here's how we benchmarked it, caveats and all: developers.redhat.com/articles/2026/…

Open models now power most production AI. On Vercel's AI Gateway, open weights went from under 10% of tokens in late 2025 to nearly 80% now. So teams are asking: why not self-host?

Then the real question: vLLM or SGLang? A 🧵:

Hola, Madrid. vLLM and llm-d are coming to town: Red Hat and @nvidia are hosting an evening on open-source inference, with @vllm_project, LLM Compressor, vLLM-omni, safe agents, and llm-d on Kubernetes.

November 12 at MOA Institute. Register: luma.com/gl1d2loa