Twitxy Saved

Patrick Wendell

Verified account This account is verified. Learn more

@pwendell

Co-founder and VP of engineering @databricks.
San Francisco Joined June 2009
595 Following 11K Followers 1.1K Posts

Latest posts 5

Patrick Wendell Verified account @pwendell
A few people asked me to comment on why Astra/Fable usage seems quite low in this new dataset.

I can't know for sure, but the #1 thing I hear from customers is that Opus/Sol/OSS models have simply gotten "good enough" for a huge fraction of enterprise work, especially when combined with routing.

developers.databricks.com/ai-index
developers.databricks.com Databricks AI Index | Databricks Developer The Databricks AI Index tracks enterprise AI from anonymized Unity Gateway traffic: which models companies use, where spend goes, how coding agents compare.
7 replies 10 reposts 118 likes 28K views View on X
Patrick Wendell Verified account @pwendell
1/ Today we’re publishing the Databricks AI Index, showing trends from enterprises who manage AI with our Unity Gateway. Data covers >10,000 companies with broad representation (Digital Natives, Fortune 500, Global 2000, etc...). Enterprise AI consumption is changing quite a bit:
33 replies 165 reposts 679 likes 138K views View on X
Patrick Wendell Verified account @pwendell
Crazy few weeks for model releases! Our findings @Databricks show several new models meaningfully advance the pareto frontier. Results below (online workload analysis of N=2,400 engineers, plus offline evals):

1. Two of the three models released last week clearly expand the cost/quality frontier: Opus 5.5 and GPT-6 Luna.
2. Opus 5.5 is now the highest quality mid-tier model. It is better than all prior Opus models, better than GPT-6 Sol, and better than GPT-5.6 Sol.
3. Opus 5.5 reduces same-task costs consistently by 20% in both offline and online analysis. This is against a baseline of Opus 4.8, the prior least-cost Opus model (Opus 5.0 was a bit of a dud with high costs and barely noticeable quality improvements).
4. Due to best-in-class quality and lower costs, Opus 5.5 is a strong candidate as an “every day default” model for coding, and we are now encouraging it for this purpose at Databricks.
5. GPT-6 Luna is very, very, very cheap. It was at least 20 times cheaper per-task than Opus 5.5 in every offline benchmark we tested and in observed online use.
6. GPT-6 Luna is surprisingly capable given how cheap it is. On one of our most difficult evaluation suites it roughly matches Opus 4.6 performance, while being 99.3% cheaper per-task than Opus 4.6 was at that time. That's a 100X cost reduction in ~9 months! This finding is preliminary and we are still evaluating Luna quality on a broader set of offline and online tests.

Our production setup: Unity Gateway to route workloads across models and trace agentic interactions. A mix of end-user harnesses including: Omingent (meta-harness), Claude Code, Codex, and Cursor.
20 replies 40 reposts 218 likes 111K views View on X
Patrick Wendell Verified account @pwendell
Today we rolled out Astra to every engineer at Databricks (N=~3500). Some notes that may be helpful to others:

1. Astra unambiguously out performs our previous highest-end models (Opus 5, Sol 5.6) on highly complex tasks, especially those related to high level system design or long range horizontal tasks.

2. Engineers given Astra increased overall coding spend by around 60% compared to baseline.

3. It is not clear Astra meaningfully improves on medium/low complexity coding tasks compared to earlier models. We suspect those tasks are mostly saturated (i.e. perfectly executed) by existing models.

4. We learned above by piloting Astra with around 200 users to gain signal on both quality and cost. We use Unity Gateway to do cohort-based experiments for all new models.

5. We give engineers a sub-budget specific to Astra to encourage them to use Astra selectively on complex tasks while preferring lower cost models for everyday tasks. Our engineers are able to mix-and-match tools and models within their overall budget envelope (we also allow for increased budgets through various mechanisms). These budgets are defined in Unity Gateway and regularly revisited.

Note: We do not have robust comparisons of Astra-vs-Fable because we have net yet rolled out Fable widely due to data retention policies.