brian stevens

Introducing llm-d on stage at Red Hat Summit was truly a privilege ...

And was great to see the Red Hat and Google effort announced by my friend the brilliant Amin Vahdat.

Today it become official, Neural Magic now a part of Red Hat.

I’m thrilled to announce that Neural Magic has signed a definitive agreement to join forces with Red Hat, Inc.

At Neural Magic our vision is that the future of AI is open, and we have been on a mission to enable enterprises to capture the powerful innovation from AI, while at the same time being free of friction and restriction. Open-source models, optimization algorithms, and inferencing systems are the heart of this - enabling enterprises to own their own AI models, privately customize them to their datasets, and deploy them on their private multi-vendor infrastructure - whether cloud, datacenter, or edge.

For the last 5 years, we have been contributing open research, open-source model quantization and sparsification tools, and open-source pre-optimized models to the community. As part of our increasing shift to support GPUs, we have been recognized as the top commercial contributor to the amazing vLLM project.

There is a saying in the startup world that an acquisition is an exit. This is not. It is an entrance. A beginning to create an open source world for AI, where an open ecosystem flourishes, and customers and the community are the benefactors.

Red Hat shares this vision, and together we will advance the world of truly “open” AI.

It’s a great day for open source AI.

Read the Press Release: redhat.com/en/about/press…

Quantization of LLM models is critical for efficient deployments. But how to avoid any negative impact of quantization on model capability? Our latest research across Llama variants will serve as a great guide.