Graphcore Research

11 billion parameters. On a phone.

We wanted to see whether Llama-3.2-11B-Vision-Instruct — far too large for a phone in its original form — could fit into a ~4 GB budget.

The answer: yes. Here's how 🧵

A tiny implementation detail in low-precision arithmetic could be biasing your AI training 😲

This interactive deep dive from Graphcore Research's @Awfidius uncovers a subtle failure mode in stochastic rounding that only appears when randomness is limited, and how it can be addressed with one simple fix 🎲

Check it out in the link below! 👇

Would you rather use 1 million × 16-bit weights, 4 million × 4-bit weights, or even 16 million × 1-bit weights?

In joint work between Aleph Alpha Research and Graphcore, we asked this question of LLMs — the answer encouraged us to embrace the wonder ✨ of 1-bit weights, which can outperform 4-bit and 16-bit weights on a fixed weight memory budget.

In our work
- ⚖️ A scaling laws evaluation prompts us to consider very low-bit formats
- 📈 Scaled-up tests show the power of memory-matched models with 1-bit weights
- ⚡ Kernel benchmarking demonstrates their feasibility for autoregressive inference

Read all about it in our blog and paper (link below! ⬇️)

Massive thanks to our collaborators at Aleph Alpha Research!

Authors: @SohirMaskey, Constantin Eichenberg, @atomicflndr and @douglasahorr

Our picks for November’s Papers of the Month are here. Out of our shortlisted papers, we spotlight three looking at LLM efficiency from different angles!

📊 First up, How to Scale Second-Order Optimization is studying optimal tuning of second order optimizers such as Muon.

🌫️ In Intelligence per Watt, the authors discuss our favorite metric on large language models: energy efficiency. And how to take advantage of edge AI inference.

🕸️ Finally, Int vs FP: A comprehensive study of fine-grained formats is contributing to an old-timer topic in quantization: integer vs floating (block) point formats.

Check out our summaries 👇

Come to the Frontiers in Probabilistic Inference today at #NeurIPS2025 to see Graphcore researcher Michael Pearce presenting "Variational Entropy Search is Just 1D Regression"