Twitxy Saved

michelle πŸ‘»

Verified account Stripe

@hazelcough

soft-bread engineer @stripe 🍞; public transit enthusiast πŸšƒ
Oakland, CA breakingchange.blog Joined October 2011
258 Following 3.8K Followers 687 Posts

Latest posts 5

michelle πŸ‘» Verified account @hazelcough
On this slow news day, we just launched Stripe evals to measure how well coding agents build real Stripe integrations.

A few cool (to me) learnings:
β€’ Our plugin helped every harness/model pair we tested. All 14 configurations scored ~3–8 percentage points higher with Stripe’s recommended skills and tools.
β€’ Agents searched less and wrote fewer tests with the plugin. Web searches dropped 96% and (often extraneous) test code dropped ~40%, while scores improved.
β€’ We used another agent to role-play the developer. The role-player starts by sharing its imperfectly-stated goals, asks follow-up questions, and works through a few rounds of conversation with its coding agent.
β€’ The last mile is hard. Agents satisfied ~78% of criteria on average, but only ~29% of runs met every required criterion.

Results and methodology: stripe.dev/evals
40 replies 13 reposts 105 likes 13K views View on X
michelle πŸ‘» Verified account @hazelcough
It makes sense intuitively that velocity eventually becomes a form of quality. Fast, iterative cycles mean more contact with reality and more chances to get it right.

But then fear stops us before we get there! We fear the quality dip in the middle of the U-curve[0] as we destroy the slow processes that protect our quality bar and create better systems that help us move faster.

I suspect this is why many organizations get stuck on the left side of the diagram as they try to transform how they work with AI.

Curious how others are finding a reliable path through. (We have some early successes, but not enough to derive a playbook.)

[0] J-curve, perhaps, since it’s plausible the future will be brighter than the past.
Kevin Yien Verified account @kevinyien Nov 6
the reason everyone gets confused on the "tension" between quality vs velocity
View quoted post on X
michelle πŸ‘» Verified account @hazelcough
My first project as a new grad at @stripe was a service to turn chargeback evidence submitted through our API into PDFs and fax them to our banking partners. I'd never seen a fax in my life.

It was nobody's dream project, and I was sure I'd get fired, so I went deep. I learned how much complexity we could hide behind a simple API, even when our banking partners still relied on paper faxes. Two years later, I got to build the first version of Stripe Radar to stop fraudulent chargebacks in the first place.

You often don't get to choose where you start, but going deep, becoming the expert, and earning the trust to take bigger swings is still a great way to build a career. AI has changed how we work, but I don't think it's changed that playbook.

@vikrams039 wrote the guide I wish I'd had as a new grad: vikramsingh.me/increase-your-…
www.vikramsingh.me Increase your slope A new grad’s guide to becoming effective at Stripe