Cindy Sridharan (@copyconstruct)

Pinned post

How to effectively debug with agents when you no longer read or understand the code

In this post, I propose a debugging framework for agents that doesn’t rely on human knowledge of code to get to the right “root cause”

The TL;DR are the screenshots.

Link to post in the reply.

Cindy Sridharan @copyconstruct Sep 14

Writing a post about “how to effectively debug code when you no longer fully understand it”.

This is real bug I had to fix in ~9K LOC (100% AI generated).

None of the tactics I started with here helped “root cause” the issue, lol. Had to take a different approach to isolate it.

View quoted post on X

Cindy Sridharan @copyconstruct 6d

CI as it currently exists (build all the code to run the full test suite) seems like a waste of compute, which can be better assigned to agents to analyze and test select parts of code but way more rigorously.

My pre-merge tests don’t do full builds; just targeted micro-tests.

Lucio Franco @lucio_d_franco 6d

@copyconstruct @poteto How do you deal with the CI time to run all those tests, etc?
View quoted post on X

Cindy Sridharan @copyconstruct 6d

Revisiting my tweets from 6m ago is funny, because I don’t run into any of these problems now at all.

What was missing in my setup 6 months ago was the verification process I now use.

70% of my daily token spend goes into verifying patches, and only ~20-30% into creating them.

Cindy Sridharan @copyconstruct May 15

Might be anecdotal, but just this week multiple AI generated PRs with subtle bugs got merged that required several additional days and a lot of manual verification to fix.

“Velocity” isn’t going up when you consider how much effort was spent fixing things post-merge, pre-deploy.
View quoted post on X

Cindy Sridharan @copyconstruct Nov 5

This is 100% true.

Your reliability in general should be *higher* with agent written code, as measured by error rates, SLO violations, client regressions etc.

If you’re having more or equal incidents now than when you were manually writing code, that’s likely a “skill issue”.

David Crawshaw Verified account @davidcrawshaw Nov 5

@thorstenball Agreed. We have shockingly few production errors these days. E.g. panic from a missing nil-check happens once every couple of months, and usually involves legacy code humans partially wrote. At this point, if I write it by hand you are going to get more bugs.
View quoted post on X

Cindy Sridharan @copyconstruct Nov 3

How to design good verification loops?

Well, that’s most of the job now, isn’t it?

Last week, I finally landed a refactor I started in Jan. It’s good I waited b/c the Jan version was buggy af.

This is the verification process I used for refactors (have different ones for perf)

Yugander @ykrishansrangra Nov 3

@copyconstruct How does one ensure verification loops are actually correct? Rigorous needs to take cost into consideration as well.
View quoted post on X

Cindy Sridharan

Latest posts 5