Twitxy Saved

Matt Johnson

Verified account This account is verified. Learn more

@Metahertz

Bringing Data Back | Literal Hacker | Wannabe Pilot | Car Guy | Change one thing or it's not debugging.
Manchester, UK medium.com/@metahertz Joined July 2009
697 Following 997 Followers 4.1K Posts

Latest posts 5

Matt Johnson Verified account @Metahertz
This is what I've been saying about tuned models for specific tasks. Bigger !=Better.
Scott Sanchez Verified account @scottsanchez Aug 31
One thing about running local models: they're lacking great web search. There are a ton of options, but useful benchmarks didn't exist. So I ran a bake-off this weekend...

I tested 4 leading open-weight models: @deepseek_ai (V4 Flash), @Zai_org (GLM 5.3 Flash), @Alibaba_Qwen (3.8 27B & Flash Next) each with 10 search provider tiers from @brave, @ExaAILabs, @p0, @serperapi, @tavilyai.

2,000 graded answers, real money paid.

Clear winner: Qwen 3.8 27B + Parallel Turbo. 50/50 correct answers, 5.7s per answer, $1.26 per 1k questions.

Full data: local-search-bench.vercel.app

All tests were run on @NVIDIAAI DGX Sparks via @MiaAI_lab's recipes, orchestrated and judged by me using Fable 5 in @AnthropicAI Claude Code.
View quoted post on X
Matt Johnson Verified account @Metahertz
I'm running Qwen3.8-Flash-Next 320B (Yes, 320B!) on £700 of hardware at home at full 262K Context:

Is it conversational quick? Nope. But behind a harness for deep planning/thinking before letting 3.8 30B take over on another node, exciting!

Context/slot - 262,144
VRAM - 43.3 GB

Qwen3.8-Flash-Next, 320B, Q4_K_XL 262K ccx
4× Tesla P100 12GB, 2× Xeon E5-2650 v4, 187 GB (RAM can be reduced ~120GB)
13.98 tok/s Decode (15 @ 32K Context)
22.79 tok/s Prefill ( 25 @ 32K Context)

Running on a Dell 1U ~10Y/O Server via #Proxmox with all 4 GPU's VT-D Passthru.

#LocalAI #Qwen