David Dalcu

9B Model one shot these !
It's still cooking, probably another few days.
Small specialized models are the future, please experiment !

Have you ever seen a 9B Pagoda ?
Trying to distill Deepseek into Qwen 9B, not bad....
I hope the new Qwen drops today, and makes all this work pointless !

MLX-Serve 26.11.2 is out
⭐️ github.com/ddalcu/mlx-ser…
Many ways to run Deepseek from ~80gb - 512 ram.
@ivanfioravanti 's RDMA code is in for the lossless quant + Massive M3 Speedups
@Beamsters1 SushiSeek STQ
@TheDavidTai 's EXL3 quants
@UnslothAI or GSQ-RCO GGUF Quants

Thank you:
elkaix, @fedeandri, @garnetlyx, @hyspace, jondestoppeleire, lborloz, pcoppinger, sjordan0228, slava-kudzinau, @STRML_ , toxeh, trigger2k20, @yoyo930021

This is crazy, a local model running on my Mac Studio... 🤯 ...100% Local, @deepseek_ai iQ-MLX-3.0bpw + MLX-Serve + @pidotdev + @threejs

Coming soon, more ways to run Deepseek sub 128GB Ram on MLX-Serve.
Honestly this model is so good, at 96 - 128GB, It should be everyone’s new default.
Upcoming pruned iQ-MLX DS4 pack will run on a 96GB M5U with ioGPU set to 90, it will be tight, but fully in memory.
And... hoping for a Qwen's release soon, and it will make this post obsolete 🤞