How to turn slop into gold. Super concrete steps from awesome @poteto on making coding agents trustworthy:
1. Clean up the code your agents learn from.
Your codebase is one of their biggest inputs. Agents copy whatever looks like an established pattern, including workarounds, deprecated code, and yesterday’s questionable decisions. Clean those up, or watch them reproduce.
2. Turn corrections into enforceable checks.
Every time you correct an agent, ask: can this become a lint rule, a type constraint, a test, or a better API? Put the correction somewhere the system can enforce it. A growing AGENTS.md still requires the agent to follow every instruction.
3. Remove opportunities for predictable mistakes.
Her team @SpaceXSI restricts React’s useEffect to framework internals because agents repeatedly misuse it in product code. They also enforce import boundaries between application processes. These constraints eliminate whole categories of mistakes that would otherwise keep coming back in review.
4. Give agents a reliable way to run and inspect the product.
Provide debugging tools, reusable CLIs, and a “feature map” describing the application from the user’s perspective. Now “that thing in the left sidebar” has a shared meaning, and each agent can reuse the tools for checking it.
5. Review evidence of behavior.
Have agents produce screenshots, videos, and state diagrams. These help you check what the product actually does and how closely that matches your intent. A convincing explanation of a fix is easy to generate; seeing the fix work is considerably more useful.
6. Spend serious effort on verification.
Her team uses runtime verification and is experimenting with property-based UI testing and formal methods, including TLA+ model checking. Lauren says most of her token usage goes toward verification. Agents make some previously expensive testing work much more practical.
7. Automate once that loop is reliable.
Her team now has agents exercise new releases, find bugs, open fixes, and merge them automatically. Getting to that level of trust took her about six months of investing in the environment.
The results she reports: 11,000 pull requests shipped by her team in a month, a revert rate around 0.5%, and improvements in startup, typing, and switching chats.
And what about infamous “software factory”?
if your coding loop produces slop, automating it produces more slop, faster, with a larger token bill.
Start by making good code the easiest thing for an agent to produce. Then scale it.
As always, content at @aiDotEngineer is gold 🏆
1. Clean up the code your agents learn from.
Your codebase is one of their biggest inputs. Agents copy whatever looks like an established pattern, including workarounds, deprecated code, and yesterday’s questionable decisions. Clean those up, or watch them reproduce.
2. Turn corrections into enforceable checks.
Every time you correct an agent, ask: can this become a lint rule, a type constraint, a test, or a better API? Put the correction somewhere the system can enforce it. A growing AGENTS.md still requires the agent to follow every instruction.
3. Remove opportunities for predictable mistakes.
Her team @SpaceXSI restricts React’s useEffect to framework internals because agents repeatedly misuse it in product code. They also enforce import boundaries between application processes. These constraints eliminate whole categories of mistakes that would otherwise keep coming back in review.
4. Give agents a reliable way to run and inspect the product.
Provide debugging tools, reusable CLIs, and a “feature map” describing the application from the user’s perspective. Now “that thing in the left sidebar” has a shared meaning, and each agent can reuse the tools for checking it.
5. Review evidence of behavior.
Have agents produce screenshots, videos, and state diagrams. These help you check what the product actually does and how closely that matches your intent. A convincing explanation of a fix is easy to generate; seeing the fix work is considerably more useful.
6. Spend serious effort on verification.
Her team uses runtime verification and is experimenting with property-based UI testing and formal methods, including TLA+ model checking. Lauren says most of her token usage goes toward verification. Agents make some previously expensive testing work much more practical.
7. Automate once that loop is reliable.
Her team now has agents exercise new releases, find bugs, open fixes, and merge them automatically. Getting to that level of trust took her about six months of investing in the environment.
The results she reports: 11,000 pull requests shipped by her team in a month, a revert rate around 0.5%, and improvements in startup, typing, and switching chats.
And what about infamous “software factory”?
if your coding loop produces slop, automating it produces more slop, faster, with a larger token bill.
Start by making good code the easiest thing for an agent to produce. Then scale it.
As always, content at @aiDotEngineer is gold 🏆
