Jonathan Spangler

Eigen Labs

@Spangler3000

Pursue the unknown | Eigen Labs / Darkbloom , USMC Vet

San Diego, CA github.com/jonathan308 Joined November 2023

304 Following 538 Followers 876 Posts

Latest posts 5

Pinned post

Jonathan Spangler @Spangler3000 Sep 27

@jundotkim , I published a bunch of PR's for OMLX for m5U optimization for Mimo 2.6 flash , GLM 5.3 flash and qwen 3.8 Next .

As always Lossless by design no tricks or benchmax
BLUF : M5 ultra is insanely fast . 🔥

The numbers are pretty wild after a day and a half of lossless optimizing.

Qwen3.8-Flash-Next Prefill: 5.6k tok/s @ 8k–16k · 5.5k @ 64k · 4.5–4.8k @ 4k Decode: 109–124 tok/s (MTP) Peak mem: ~109 GB

GLM-5.3-Flash Prefill: 2.35k tok/s @ 16k & 64k · 2.32k @ 4k Decode: 63 @ 1k · 59 @ 4k–16k · 56 @ 64k (no spec dec) Peak mem: ~187–191 GB

MiMo-V2.6-Flash Prefill: 3.3k tok/s @ 8k Decode: 99–118 @ 8k · 89–99 @ 1k (MTP) Peak mem: ~170–173 GB

Jonathan Spangler @Spangler3000 1d

Qwen 4 is on the horizon …. Fingers crossed for an updated flash . @QwenDevs knocked the sizing out of the park with 3.8 flash .

Can’t wait 👀👀🔥

Jonathan Spangler @Spangler3000 Oct 28

So with the price of the 512gb Mac Studio , why not just cluster 2x 256?

Jonathan Spangler @Spangler3000 Oct 15

Join the competition ! Many machines can compete in this . Can't wait to see how far you all push it.

Kydo Verified account @0xkydo Oct 15

MLX.fast and CUDA.fast for Bonsai 2 9B is live!

Bonsai 2 9B a 2GB model; so, many many machines can optimize them locally. The Bonsai model we did had 5x in performance increase and I cannot wait to see what the community does with this one!

View quoted post on X

Jonathan Spangler @Spangler3000 Oct 15

Try out bonsai 9b !

Kydo Verified account @0xkydo Oct 15

We're excited to offer day-one support for the new Bonsai 2 model.

You can try Bonsai 9B at console.darkbloom.dev. It's a 2GB model that handles many everyday and coding tasks. The first 100 people get 250M free tokens on us to try it out.

For the optimizers, you can also improve Bonsai's performance at mlx.fast and cuda.fast to make it even faster on local machines.

Bringing more capable intelligence to more devices will be a massive unlock for the next stage of local AI. PrismML's work is a critical piece of the puzzle, and at Darkbloom, we're excited to support them from day one. Thank you to the @PrismML team for trusting us.
View quoted post on X