Everyone saw Flapping Airplanes' valuation and was like its might morphing mega kernel time !!
driss guessous
@drisspg
306 Following 1.7K Followers 713 Posts
Latest posts 5
they're formalizing me at work tomorrow
Fun Fact you dont actually need a per request kv cache for the first layer of your Transformer. I had a convo about this with Richard Zou a while ago finally got around to prototyping
0:18
TIL: the 16x2 atomics can be > 2X times faster than single element version. And that the codegen goes from a true 32 bit add (ATOM.E.ADD.F16x2) to a CAS loop
gist.github.com/drisspg/a887c7…
gist.github.com/drisspg/a887c7…
gist.github.com Narrow (16-bit) vs wide (packed 32-bit) half/bf16 atomicAdd microbenchmark (pytorch#202905) Narrow (16-bit) vs wide (packed 32-bit) half/bf16 atomicAdd microbenchmark (pytorch#202905) - README.md World first; guest blog post on nuggets :)
drisspg.github.io/nuggets/TorchI…
By the great Elias Ellison
drisspg.github.io/nuggets/TorchI…
By the great Elias Ellison
drisspg.github.io TorchInductor Walkthrough A guest notebook by Elias Ellison: torch.compile's IR stages, how the Inductor scheduler decides what to fuse, and newer scheduler passes, with real logs from every cell.