jietang

GLM-5.3 is on T-Systems now....

GLM-5.3 is on Amazon Bedrock! docs.aws.amazon.com/bedrock/latest…

Two weeks.

That's how long it took to go from GLM-5.3-Flash's first run on domestic accelerators to serving all of its production traffic, with 3.2× end-to-end throughput along the way.

What I keep thinking about is who did much of the work: an Infra Agent powered by GLM-5.3.

A model helping optimize the system that serves it.

The conditions were hard. Limited memory and interconnect bandwidth. 1M-token context. Multimodal requests. An immature software stack where kernels were missing and documentation was often guesswork. Every optimization was a trade: compute for memory (ReplaySSM), communication for memory (intra-node tensor parallelism), precision for capacity (mixed INT8/FP8/BF16 caching), and disaggregation for scheduling freedom (Encode–Prefill–Decode).

But the most important lesson wasn't about any single optimization.

When the agent got stuck, it was rarely because it couldn't write the code. It was because it didn't know *why* things got worse.

"Throughput down 20%" tells you something broke. It doesn't tell you which layer, which hypothesis, or what to test next. In RL terms, it's a sparse reward with a credit assignment problem. And an end-to-end benchmark that takes hours makes exploration painfully slow.

Senior engineers solve this with an implicit process reward in their heads. They know when to check the timeline, when to run a microbenchmark, and which layer's output to compare.

So we made that explicit. We call it dense feedback: layered verification interfaces the agent can call directly.

Correctness feedback: did it compute right?
System behavior feedback: where did the time go?
Performance feedback: which option wins, under which conditions?

Each signal has to be local, cheap, and objectively verifiable.

Three things the agent found:

First, precision drift in KDA's context-parallel path that grew with sequence length. The cause was TF32 rounding error compounding through chained state-matrix merges. The fix is now merged upstream in Flash Linear Attention (PR #1180).

Second, KV transfer never overlapped with DeepEP dispatch. The agent followed the call chain across the Python/C++ boundary and found that the intranode path never released the GIL. After the fix, transfer overhead fell from over 30% to under 1%.

Third, a decode kernel recomputing the same normalization four times because of how it was chunked. The agent restructured it and got a 1.71× speedup. The idea came from "optimization skeletons" it had distilled by reading existing kernels across SGLang, FLA, and DeepGEMM.

To be clear about the boundaries: humans still defined the goals, built the feedback environment, and reviewed every high-risk change.

But the engineer's role is changing, from the person who solves the problem to the person who designs the feedback.

There's a deeper implication too. A layered, verifiable feedback environment built on real infrastructure tasks is exactly what training the next generation of models needs most. Every task the agent completes can become training ground for its successor.

We are still far from recursive self-improvement.

But the smallest loop now exists.

The model optimizes the system. The system serves the model.

Add oil and Happy Teacher's Day

Are you sure? Finding the optimal model size is tricky: data volume, active parameter count, the number of environments, and the target inference cost. Model performance also depends on many other factors, each introducing its own variability.