Pinned post
this was lots of fun and lots of all nighters over the past few weeks. really happy with what we achieved!
NVIDIA Blackwell can achieve 303 output tokens/s for DeepSeek R1 in FP4 precision, per our benchmarking of an Avian API endpointView quoted post on X
Artificial Analysis benchmarked DeepSeek R1 on an @avian_io private API endpoint. Running DeepSeek R1 in FP4 precision on NVIDIA Blackwell, their endpoint achieved 303 output tokens/s - the fastest speed we have measured yet for DeepSeek R1.
The FP4 version of DeepSeek R1 maintained accuracy across our evaluation suite as compared to the native FP8 version.
Inference speed is especially critical for reasoning models that ‘think’ before they answer - we look forward to wider availability of NVIDIA Blackwell hardware in the coming months!