Autoregressive models extend a response one token at a time. Masked diffusion models can predict multiple positions together, then repeat that process to complete the response.
Both can use transformer components such as embeddings, attention, and feed-forward layers. Their attention patterns and training objectives differ.
1) Autoregressive LLMs
In standard autoregressive decoding, each forward pass produces a probability distribution for the next token.
The selected token is appended, and generation continues from that longer sequence.
Causal attention prevents a position from accessing future positions. Earlier token representations therefore stay unchanged as new tokens arrive.
That allows their KV to be cached and reused.
A 500-token response normally requires 500 sequential token-generation steps, although techniques such as speculative decoding can reduce expensive model passes.
2) Diffusion LLMs
In the masked diffusion approach shown below, generation starts with masked output positions alongside the prompt.
A denoising pass predicts candidate tokens for multiple positions using bidirectional context.
With confidence-based sampling, selected predictions are retained while other positions remain masked for subsequent steps.
The model progressively fills the sequence. It doesn't necessarily produce a complete draft and repeatedly rewrite every token.
Some implementations generate block by block rather than processing the entire response together.
Multiple tokens can be finalized per step, allowing fewer steps than output tokens. The step budget affects both speed and quality.
There are some engineering tradeoffs.
Bidirectional attention makes standard autoregressive KV reuse harder. A token's representation can change when surrounding positions are updated, even if that token itself stays fixed.
Caching therefore depends on the architecture and decoding strategy. Block diffusion, for example, can support KV caching across completed blocks.
Fewer steps also don't guarantee lower latency. Each step may process many positions, so total computation and implementation matter.
Bidirectional conditioning also supports filling gaps in text or code.
I covered how KV reuse works in autoregressive decoding in a separate breakdown.
It covers what gets cached during prefill, how the cache grows during decoding, and why it becomes the main memory bottleneck at long context.
We package repeated workflows as skills all the time, but the agent still re-reads the skill and decides every step on every run, which costs tokens and time.
Devs just open-sourced Swamp, where your coding agent turns a repeated skill into a workflow that it builds once and runs again and again.
You describe the task once, and the agent breaks it into reusable behaviors connected in a workflow.
Every behavior has typed inputs and outputs, and plain code decides which step runs next. Swamp then gives you:
- model calls only on steps that need judgment - secrets pulled from vaults and redacted from output - versioned data from every run that you can query later - a full record of what happened on each run
Swamp plugs into the coding agent you already use, like:
- Claude Code - Codex - Cursor - OpenCode - And 5 more agent harnesses
In Swamp, the model is a step in the workflow, not the thing deciding what happens next.
The model steps still reason like any LLM call.
However, in this case, code owns the loop, so the agent doesn't spend tokens rediscovering the same process every time.
Here's an official result, where they turned a code-review skill into a Swamp workflow:
- Skill in Codex: about 4.6M tokens, 23 agents, 12.8 minutes - Swamp workflow: about 506K tokens, 3 agents, 6.6 minutes
That is ~8x fewer tokens in about half the time, and most of the savings came from replacing the coordinator agent with plain code.
You can keep using skills while you prototype. Once a task starts repeating, ask your agent to turn it into a Swamp workflow.
This helps answer practical questions such as:
- Which steps actually need a model? - What exactly happened on last week's run? - Can a cheaper model handle the one step that needs judgment?
Once it's built, you can run the workflow from your agent, the CLI, a schedule, or an API, with every run recorded.
So you are no longer paying a model to figure out the same workflow on every run.
Instead, you spend tokens only on the steps where the model's judgment adds something.
The animation below shows how Swamp differs from Skills, and I worked with the team to put this together.
If you want to understand the coordination layer that Swamp replaces with code, I wrote a full breakdown of how Claude Code splits work across subagents and agent teams.
It covers when a single subagent is enough, when you need a team of agents that coordinate with each other, and what that coordination costs in context and tokens.
They can now do the one thing they used to need you for.
When you ask an agent to build an app, it can write the schema, the routes, and the data layer in a few minutes.
But then it needs a database, and everything stops since someone has to open a browser, create an account, provision a database, set up a user, and paste the connection string back into the session.
Those steps were built for a human, so the agent just waits for you to complete them.
MongoDB just solved that with Atlas Ephemeral Clusters.
Now, your agent can make one API call, and Atlas returns a live database in under a minute.
Your agent saves the connection string, allowing you to keep building on a real MongoDB Atlas cluster.
The part I like most is that the agent can create the database cluster and work with the data, but only a human can make the decision to claim it and take ownership.
If you don’t claim it, the cluster pauses after 48 hours and deletes itself after 7 days, so abandoned experiments clean themselves up automatically.
If you claim it, it becomes a permanent free Atlas cluster with the same data and the same connection string.
So the agent handles the setup end-to-end, and you only step in once the project is worth keeping.
Atlas Ephemeral Clusters are in public preview, and the docs shared above walk you through the full flow.
The video below shows this in action, and I worked with the MongoDB team to put this together.
If you are building with coding agents, the next step is getting several of them to work together on one project.
I wrote a full breakdown of how Claude Code handles that with subagents and agent teams.
It covers when a subagent is enough, when you need a team of agents that coordinate with each other, and how each setup manages context and hands off work.
Instead of stacking more layers, it runs the same layers multiple times so a 4B model can think like a 9B one.
Here's how it works:
In a standard transformer, every token passes through a stack of decoder blocks once, and each block has its own weights.
So if you want a deeper model, you add more blocks, and the parameter count grows with the depth.
NVIDIA's Dory1.0-4B decouples the two by reusing a chunk of its layers.
The model is split into three parts:
> An input block of 8 blocks runs once and turns the tokens into a hidden state x.
> A recurrent block of 20 shared blocks runs k times, where k can be anywhere from 1 to 4.
> An output block of 8 blocks runs once on the final state, followed by the LM head.
Before every pass through the recurrent block, x is added back to the current state and the sum is L2-normalized.
This keeps the original token representation in view on every pass, instead of each pass only seeing what the previous one produced.
Both models in the diagram below store 36 blocks of weights.
The standard transformer applies each block once, while Dory applies 36 blocks at k=1, 56 at k=2, and 96 at k=4, i.e., 72, 112, and 192 physical layers from the same checkpoint.
The extra depth is actually helpful on reasoning and agentic tasks:
> SWE-bench Verified:
- at 1 loop → 50.3 - at 2 loops → 59.5 - at 4 loops → 61.8
> 12-benchmark average:
- at 1 loop → 35.17 - at 2 loops → 39.67 - at 4 loops → 40.60 - and with 2 or more loops, it beats Qwen3.5-9B, Gemma 4 12B, and Gemma 4 26B-A4B.
More loops implies more compute per token, but the 4-loop model also writes 1.3x to 2.5x fewer output tokens on agentic tasks.
So 2 loops end up costing only 1.03x to 1.25x the compute of 1 loop end to end, which is why NVIDIA recommends 2 as the default.
By default, Dory keeps a separate KV cache for every loop, so at 4 loops the recurrent layers store 4x the cache.
To avoid this, it can keep only the cache from the last loop, which uses the same KV memory as a normal model with only a small change in accuracy.
One thing to note here is that looping helps the model reason over what it already knows, but it does not add knowledge.
For instance, on tasks like AA-Omniscience, accuracy stays at 10.0 whether you run 1 loop or 4, and on knowledge-heavy tasks like GPQA, Dory still trails Qwen3.5-4B.