Twitxy Saved

Adam Gastineau

@iam_agg

Currently hacking on the Humane Ai Pin (previously Vision Pro and Analogue Pocket). Mastodon: fosstodon.org/@agg23
github.com/agg23 Joined May 2021
238 Following 2.5K Followers 1.2K Posts

Latest posts 5

Adam Gastineau @iam_agg
Have people settled on a location for sharing LLM speed benchmarks? The sites I find seem abandoned

It's frustrating not being able to compare. I can't tell if M5 Ultra Q8, 32k, single stream at 185 t/s is good or not. Some places say 6000 Pro isn't typically hitting that at Q4
Adam Gastineau @iam_agg
I wanted to try messing with (prompting) inference improvements with M5 Ultra. I'm rather surprised at the results and after a lot of checking I think they're accurate

M5 Ultra 256GB, 80 core, Qwen3.8 Flash Next Q8

Up to 85% faster decode vs oMLX main: 378 t/s @ 32k, 8 streams