Bohan Hou (@bohanhou1998)

Bohan Hou @bohanhou1998 Nov 4

After our kernel-agent experiments, we had to confront an uncomfortable question: Do we still need DSLs and ML compilers when agents can do it all？

In short: We're still here! 😎

If you're curious why, here's our take:
tvm.apache.org/2026/11/03/ret…

We've also rebuilt TIRx from the ground up as a shared DSL and compiler foundation for agentic kernel development.

Bohan Hou @bohanhou1998 Sep 29

2.94× KDA fwd, 6.84× KDA bwd, 2.59× MSA prefill, 3.99× MSA decode, 1.33× KDA decode, 1.71× MLA, 1.68× VSA — just by pressing Enter.

After years of working on ML compilers and designing many abstractions, the most useful compiler thing in TIRx I’ve ever built might just be a PTX tablegen.

Hongyi Jin Verified account @HongyiJin258 Sep 29

AI agents are becoming GPU kernel engineers. We're rethinking the compiler stack around them.

Meet TIRx Harness: an open compiler harness that gives agents an environment to explore, debug, and optimize GPU kernels.

🧱 Minimal stable low-level compiler foundation for predictable GPU programming 🔍 Domain-specific compiler analysis to guide debugging and optimization 📚 Kernel zoo of 60+ kernels and hardware specs to reuse strategies and discover new ones 📊 Remote GPU evaluation for reliable benchmarks and profiling

Agent-built Kimi Delta Attention kernels achieved geometric-mean speedups of 2.94x over FlashKDA (forward) and 6.84x over FLA (backward).

Our bet: the next leap in agentic GPU programming will come from engineering the environment agents optimize in.

Check out our blog: blog.mlc.ai/2026/09/29/tir…

View quoted post on X

Bohan Hou @bohanhou1998 Aug 5

BTW, I noticed that the CuTe DSL 4.7 PR includes a static analysis tool for catching certain synchronization bugs. I expect we’ll see many more tools like this going forward. :)

Bohan Hou Verified account @bohanhou1998 Aug 5

Very interesting post. One optimistic thought:

Models will keep getting better—GPT-6, 7, 8, whatever comes next.

But if humans still need to review every GPU kernel they generate, the real bottleneck remains.

I’ve seen PRs add a single PTX instruction like fence.proxy.async to fix bugs that only appear after hours or days on hundreds of GPUs.

For example, FlashKDA relased last week: github.com/MoonshotAI/Fla…

I already run 3–5 agents in parallel.

The slowest part is still me.

I still have to read the code.

So how far are we from a truly human-free GPU kernel development loop?

And if we still need tooling around generated code—even at the PTX level—then abstractions haven’t disappeared.

They’ve just moved from the programming interface into the verification infrastructure.
View quoted post on X

Bohan Hou @bohanhou1998 Aug 5

Very interesting post. One optimistic thought:

Models will keep getting better—GPT-6, 7, 8, whatever comes next.

But if humans still need to review every GPU kernel they generate, the real bottleneck remains.

I’ve seen PRs add a single PTX instruction like fence.proxy.async to fix bugs that only appear after hours or days on hundreds of GPUs.

For example, FlashKDA relased last week: github.com/MoonshotAI/Fla…

I already run 3–5 agents in parallel.

The slowest part is still me.

I still have to read the code.

So how far are we from a truly human-free GPU kernel development loop?

And if we still need tooling around generated code—even at the PTX level—then abstractions haven’t disappeared.

They’ve just moved from the programming interface into the verification infrastructure.

github.com Fix missing proxy fences around TMA accesses by esp0r · Pull Request #25 · MoonshotAI/FlashKDA Summary Fixes the missing cross-proxy fences reported in #24 (found by auditing the SM90 forward kernels against the PTX ISA proxy model): fwd_kernel2.cuh — FP32 state store: smem_cvt_bf16_to_fp32...

Stuart Sul Verified account @stuart_sul Aug 5

(1/6) If a prompt can generate every layer of the stack, what are programming abstractions worth?

GPU programming is at a weird inflection point. For decades we stacked layers of abstraction to ease our cognitive load. This year, we find ourselves constantly deleting them.

View quoted post on X

Bohan Hou @bohanhou1998 Jul 29

Don’t feel bad about causing infra folks pain—NVIDIA does that every 2–3 years anyway :)

Horace He Verified account @cHHillee Jul 29

Presenting my grand unified theory of ML researcher impact: Your impact is directly proportional to how much pain you cause to infra.

Fundamentally, you can only inflict pain upon infra if your approach actually works. And the better your approach works the more pain infra is forced to endure.

So, to give some examples:
- MoE's add a ton of data-dependent computation => pain (shazeer++)
- GDN/KDA are the most complex architecture I've been forced to care about and a very annoying matrix inversion => pain (sonta++)
- Muon is much more annoying than Adam and causes annoying restrictions on parallelism => pain (keller/jeremy++)
- RL scaling forced many researchers to care about LLM inference and RL infra as a category => pain (tworek++)

Even papers like Attention Is All You Need have lead to significant pain! Before transformers were invented everyone was running small jobs and I never needed to think about kv-caches or 6D parallelism.
View quoted post on X

Bohan Hou

Latest posts 5