Twitxy Saved

Carson Poole

Verified account This account is verified. Learn more

@CarsonPoole

New York City poole.ai Joined May 2010
135 Following 1.1K Followers 1.3K Posts

Latest posts 5

Pinned post

Carson Poole Verified account @CarsonPoole
this was lots of fun and lots of all nighters over the past few weeks. really happy with what we achieved!
Artificial Analysis Verified account @ArtificialAnlys Apr 8, 2025
NVIDIA Blackwell can achieve 303 output tokens/s for DeepSeek R1 in FP4 precision, per our benchmarking of an Avian API endpoint

Artificial Analysis benchmarked DeepSeek R1 on an @avian_io private API endpoint. Running DeepSeek R1 in FP4 precision on NVIDIA Blackwell, their endpoint achieved 303 output tokens/s - the fastest speed we have measured yet for DeepSeek R1.

The FP4 version of DeepSeek R1 maintained accuracy across our evaluation suite as compared to the native FP8 version.

Inference speed is especially critical for reasoning models that ‘think’ before they answer - we look forward to wider availability of NVIDIA Blackwell hardware in the coming months!
View quoted post on X
2 replies 2 reposts 13 likes 3.8K views View on X
Carson Poole Verified account @CarsonPoole
in December of 2021 I saw a GitHub repo doing LoRA fine tuning. I realized immediately that it would allow for different elements in the batch to receive different fine tuned models, effectively allowing a single GPU to serve multiple specialized models with very small overhead

I spent the next hour writing a prototype and showed our customers, who were very excited by this. as far as I can tell, this didn’t wind up in the open source world for close to a year after this, and FireworksAI sent me a marketing email advertising this functionality as on October 14 of 2026
gabriel Verified account @gabriel1 Nov 11
i found the optimal first interview question, difficulty: impossible

tell me about one concrete action in your career, in a way where I can understand why it is impressive given what you knew at the time. use as few words as possible
View quoted post on X
Carson Poole Verified account @CarsonPoole
Of course that’s your contention.

You’re a first-year inference engineer.

You just read SemiAnalysis’s blog and saw megakernels coming to TPUs and AMD, so naturally you believe that’s the way to get to speed of light.

Next month, when you get your hands on an inference engineering cookbook—Baseten’s, probably—you’ll be singing from the treetops about the 500 nanoseconds of critical-path latency you save on every GEMM pipeline drain.

That’ll last until sometime next year, when you watch a pirated geohotz lecture livestream and become convinced that PDL gives you a strict superset of megakernel capabilities.

“Well, as a matter of fact, I won’t, because PDL introduces—”

“PDL introduces unnecessary stalls because its synchronization mechanism is very coarse.”

Yeah, I read that too. HazyResearch’s blog, “Look Ma, No Bubbles!” Just below Figure 2.

Do you have any thoughts of your own on the subject, or should I just point my agent at the SGLang repo and have it do it for me?
26 replies 76 reposts 1.2K likes 57K views View on X