Twitxy Saved

ned renzi

@NedRenzi

Doing hard things is a skill. Coach @EnjoyTheWork
Pittsburgh | Miami enjoythework.com Joined May 2012
550 Following 950 Followers 5K Posts

Latest posts 5

ned renzi @NedRenzi
Excited to see Geom launch publicly today. @ShannaWP and her team are building AI to automate the manual work of architectural drafting, starting with production homebuilding. This is a large and important problem and the team has the domain experience to tackle it.
ned renzi @NedRenzi
Become a model sommelier.
Lenny Rachitsky Verified account @lennysan Sep 7
My biggest takeaways from @illscience:

1. Company building will now be creating a series of loops. A coding loop turns a bug report into a fix and a low-risk production release in five minutes. Anish expects similar loops to spread from individuals to functions, business units, and eventually large parts of a company—from growth experiments to sales demos to legal and support. The key is to figure out what the agent doesn’t know and give it that context so it can complete the loop.

2. Moats are discovered, not designed. Anish uses Cursor as an example: it began as a high-engagement DAU product, then captured reasoning traces and trained its own models over time. A founder does not need a fully formed moat story on day one if the product has momentum, craft, and growing engagement. The classic moats—like network effects, scale advantages, brand, and proprietary data—still matter, but the small product decisions that create them often become visible only after the product is in the world.

3. We appear to be on the slow takeoff timeline. The case for fast takeoff always follows the same structure: everything up until now is OK, then something no one can articulate happens, then runaway acceleration. Anish doesn’t buy it. Model progress is real and faster than ever, but most problems aren’t intelligence-bound. A data center of PhDs doesn’t exponentially improve pizza supply chains.

4. The “permanent underclass” fears are a Silicon Valley dark fantasy. By almost every empirical measure, things have never been more distributed or opportunity-rich. Job postings for radiologists and programmers are at historic highs, despite years of “they’re cooked” predictions. And within the AI stack itself, rather than one winner-take-all platform, there are 20 credible players at every layer.

5. The biggest opportunity in consumer AI right now is “/loop make me happier” (not “/loop make me more productive”). Most people want to spend time, not save it. The biggest products in the world are entertainment and social, not productivity tools. Anish sees a spiritual hunger, particularly outside major urban centers where cultural institutions have thinned out. The opportunity: How do we feel more connected, more loved? How do we have fun? “We built a technology that extends our intellect and nothing to extend our soul.”

6. Three things have held consumer AI back, and all three are now improving. First, model costs were too high for free-to-use consumer products. Second, chat is a high-agency interface that works for Elon and Sam but not for the average consumer, who needs something between chat and TikTok. Third, the technology has been aimed almost entirely at productivity rather than connection and entertainment. The consumer moment is coming; Anish puts us at iPhone 2010, pre-Airbnb, pre-WhatsApp, pre-Uber.

7. Humans will remain critical for identifying the next opportunity. Agents are excellent at climbing to a local maximum, but then they plateau. They aren’t great at picking which hill to climb next. That’s where humans come in. Anish illustrates this with a chart he uses in conversations: agents hill-climb, then a human steps in to set the direction for the next climb, and the cycle repeats.

8. The most important attribute for teams is now ambition. Three years ago, Anish and his colleagues would pass on companies that seemed too crazy or complex. Today the opposite is true: an idea that’s too small isn’t worth engaging with. The firm’s internal posture with every founder: “We’re here to help you build the strongest form of your vision.”

9. There’s a big opportunity in creating very expensive consumer software. The old wisdom was that consumer products have to be free. Anish is taking the opposite position—that consumer discretionary spend is entirely up for grabs, and price is a measure of product-market fit. His product exercise for founders: what would the Birkin bag version of your product, at $1,000 or $10,000 a month, have to do to justify that price?

10. Become a model sommelier. Models are not interchangeable; Anish experiences different models as suited to different kinds of work. His way to learn their shape is to build something with every release, ideally using one or two low-stakes projects as a recurring test bed. His minimum heuristic is to ship once a week.
View quoted post on X
ned renzi @NedRenzi
A good way to evaluate and train agents.
Sheel Mohnot Verified account @pitdesi Jul 29
This is really cool.

TL;DR: Basis and Braintrust are creating an open standard for evaluating long-running agents based not only on what they accomplish, but on whether they follow a reliable process along the way.

If you don't know what that means, I'll try to explain:

Agent evals are tests that measure how well an AI agent performs. They’re important because agents can behave differently each time, and one mistake can affect everything that follows.

AI teams use evals to check whether an agent is reliable and safe enough for real-world use, and whether changes actually improve it.

Most agent evaluations focus on outcomes: Did the agent produce the correct answer and/or successfully complete the task?

That works when success is easy to verify, like a math problem against an answer, if an appointment was booked, or if a game-playing agent won a game.

But many tasks take hours or days, have hundreds of decisions, and don’t actually have a clean answer.

An agent might reach the right result while following an unreliable process... if we only grade the outcome, we lose the signal about what actually went right or wrong along the way.

Basis is creating behavior specs: reusable descriptions of how an agent should operate across an entire trajectory.

For example, a tax agent might be required to check the latest official tax rules before giving an answer.

A judge (another AI model acting as an evaluator) can then review what the agent did and determine whether it followed that rule to let teams evaluate the agent’s process even when they don’t know the perfect final answer.

Braintrust records what the agent does, evaluates whether it followed the behavior specs, and helps teams compare its performance across different models and versions.

Together, Basis and Braintrust are open-sourcing the standard so researchers can study these behaviors and use what they learn to evaluate, improve, and eventually train agents.

It’s essentially a way to grade not only what an agent accomplished, but how it operated along the way.

very cool!
View quoted post on X