turboderp
Verified account This account is verified. Learn more@turboderp_
61 Following 1.5K Followers 135 Posts
Latest posts 5
Don't know if I should spend less time coding and more on farming engagement, but either way here's a neat cat simulation.
It's not entirely accurate (still needs to model viscosity and surface tension), but it was more of a stress test anyway, made by a new 1.66 bpw quant of Qwen3.8-Flash-Next running on one RTX 5090; no experts offloaded to the CPU, only vision tower in pinned memory (negligible overhead), and embeddings on disk.
Stable inference throughout, no reasoning loops, just smooth sailing at 200-300 t/s (and a lot of compacting since one 32 GB GPU still doesn't leave a lot of room for context, even with cache quantization).
Get yours today at huggingface.co/turboderp/Qwen…
It's not entirely accurate (still needs to model viscosity and surface tension), but it was more of a stress test anyway, made by a new 1.66 bpw quant of Qwen3.8-Flash-Next running on one RTX 5090; no experts offloaded to the CPU, only vision tower in pinned memory (negligible overhead), and embeddings on disk.
Stable inference throughout, no reasoning loops, just smooth sailing at 200-300 t/s (and a lot of compacting since one 32 GB GPU still doesn't leave a lot of room for context, even with cache quantization).
Get yours today at huggingface.co/turboderp/Qwen…
0:37
Is anyone else confused why everyone's all hyped about using "decision models" to solve deterministic problems, instead of leveraging AR models' ability to implement classical algorithms?
I challenge any Jev or Jev-like harness to compete with A* in 80 lines of JavaScript. I get the selling point of "you don't need to write any task-specific code, the decision model just figures it out." But in this case the cost was all up-front: four minutes with Astra-medium, and a one-line prompt. What am I missing?
I challenge any Jev or Jev-like harness to compete with A* in 80 lines of JavaScript. I get the selling point of "you don't need to write any task-specific code, the decision model just figures it out." But in this case the cost was all up-front: four minutes with Astra-medium, and a one-line prompt. What am I missing?
0:54
I doubt I'm the first to notice this, but MiMo-V2.6-Pro seems to have a lot of redundant weights.
If you remove the expert blocks from layers 1 through 12 and the attention blocks from layers 1-4, 6 and 8-11, end-to-end KLD is largely unaffected. Their contribution to the residual is on the order of BF16 rounding errors.
Overall, it looks like you can trim away 17% of the model's ~1T parameters and still end up with largely the same model. That's six whole Qwens.
Is this known, or can anyone confirm?
If you remove the expert blocks from layers 1 through 12 and the attention blocks from layers 1-4, 6 and 8-11, end-to-end KLD is largely unaffected. Their contribution to the residual is on the order of BF16 rounding errors.
Overall, it looks like you can trim away 17% of the model's ~1T parameters and still end up with largely the same model. That's six whole Qwens.
Is this known, or can anyone confirm?
