Latest posts 2
Opened a PR on ds4: Qwen3.8 Flash Next now decodes 16 sessions in one batched Metal forward, 203 tok/s aggregate on an M5 Max 128 GB (vs one session at a time upstream), single stream +8%
github.com/antirez/ds4/pu…
@antirez @ivanfioravanti would love numbers from your bigger boxes
github.com/antirez/ds4/pu…
@antirez @ivanfioravanti would love numbers from your bigger boxes
github.com Qwen3.8 Flash Next: batched decode and MTP across sessions on Metal (16 streams: 203 tok/s aggreg... Qwen3.8 Flash Next on Metal: native batched decode across sessions (1–16 streams), batched multi-token prediction, single-stream gains, and a session-concurrency benchmark. Measured on an M5 Max (1...