Twitxy Saved

Kombai

Verified account This account is verified. Learn more

@Kombaico

Your AI design engineer
San Francisco kombai.com Joined June 2023
4 Following 2K Followers 145 Posts

Latest posts 5

Pinned post

Kombai Verified account @Kombaico
Most frontend and design benchmarks evaluate whether models produce working UIs and/or pairwise design preferences of judges.

With this benchmark, we measure something different: the diversity of design decisions a model makes when the problem leaves room for creative exploration.

We construct paired briefs that preserve the underlying UI task while changing the product context.

For example, a clinical dashboard and a pharmacy operations dashboard share related functional requirements but call for different information architectures and visual hierarchies.

Models receive no prescribed layout or visual direction.

We then compare their generated interfaces using a similarity metric that measures structural composition and information architecture, with color evaluated separately.

This allows us to measure whether a model meaningfully changes its design approach, rather than merely changing content or styling.
Dipanjan Dey Verified account @Dipanjan_Dey Oct 14
⚠️ Frontier AI models still fall significantly short of human designers on design creativity. ⚠️

Introducing Design Creativity Bench, a new benchmark evaluating AI models’ creativity on UI design tasks.

Despite rapid advances in their ability to generate functional websites and product UIs, leading AI models still struggle to make creative design decisions.

We evaluated 16 frontier models across 80 open-ended UI tasks, each adapted to two distinct product domains alongside human designer references.

Existing UI benchmarks often measure a single output quality metric. Instead, we measure three dimensions of design creativity:

- Originality: How distinct a model's designs are from other models' outputs for the same brief.
- Creative Range: How much a model adapts its design decisions when the product domain changes.
- Appropriateness: How well the generated UI satisfies the functional requirements of the brief.

The results reveal a substantial gap in overall creativity: The best model, @OpenAI GPT-6 Astra, scores 65.5 against a human reference of 92.3.

We're publishing the full leaderboard, methodology, and reproducible evaluation framework to make design creativity a measurable dimension of frontier model capability.

📊 Benchmark Leaderboard: kombai.com/benchmark/
📄 Paper: arxiv.org/abs/2610.11539
🤗 HuggingFace Datasets: huggingface.co/datasets/Komba…
🔗 Github: github.com/kombai-io/desi…
View quoted post on X
1 replies 10 reposts 20 likes 845 views View on X
Kombai Verified account @Kombaico
Hy4 Flash from @TencentHunyuan is now live on Design Creativity Bench, with a creativity score of 40.6.

Hy4 Flash is the first Flash-tier, dedicated low-cost, high-speed model in Tencent's model lineup, with a median generation time of 2.3 minutes and a median cost of $0.006 per design. That places it on the cost-performance Pareto frontier.

Compared with other low-cost models, it's:

- 14x cheaper than Hy4 Preview ($0.006 vs $0.086) and 3x faster (2.3 vs 7.1 minutes per design), with a lower creativity score (40.6 vs 46.2)

- 73% cheaper than GLM-5.3-Flash ($0.006 vs $0.022), with a higher creativity score (40.6 vs 38.5)

- 40% cheaper than Muse Ember ($0.006 vs $0.010), though Ember scores higher (47.7 vs 40.6) and is nearly 4x faster (0.6 vs 2.3 minutes per design)

Congrats @TencentAI_News @TencentGlobal and the Tencent team!

Full leaderboard: kombai.com/benchmark
Kombai Verified account @Kombaico
Qwen4 Max from @Alibaba_Qwen is now live on Design Creativity Bench with a creativity score of 45.5 (#10 of 16).

1) That’s a significant drop from Qwen3.8's 58.1. We’re seeing a similar decline on specialized benchmarks like Vals AI, where Qwen4 Max scores 41.3% on Legal Research vs. 46.2% for its predecessor.

2) Qwen4 meets more of the brief than Qwen3.8 (98.6% vs 97.6%) and is cheaper per token ($1.50/$4.50 vs $2/$6).

But Qwen3.8 had one of the most distinctive styles on our panel (0.623 originality), while Qwen 4 now scores the lowest (0.538).

3) This fits a design convergence pattern we’ve seen this week.

Fable 5.2, Gemini 4 Carbon, Gemini 4 Hydrogen, and now Qwen4 Max have all moved toward the Opus/Fable house style, losing originality compared with their predecessors.

Explore the full leaderboard: kombai.com/benchmark

@QwenDevs @AlibabaGroup @QwenAIFans
0 replies 0 reposts 15 likes 2.8K views View on X
Kombai Verified account @Kombaico
Gemini 4 Hydrogen from @GoogleAI is now live on Design Creativity Bench, with a creativity score of 49.4.

Hydrogen is the fastest Gemini 4 model we've tested, with a median generation time of 3.4 minutes and a median cost of $0.39 per design.

Compared with the other Gemini 4 models, it's:

- 1.5x faster than Argon (3.4 vs 5.0 minutes per design), but 11% more expensive ($0.39 vs $0.35).
- 2.3x faster than Carbon (3.4 vs 7.9 minutes per design), and 24% cheaper ($0.39 vs $0.51).

Congrats @OfficialLoganK @demishassabis and the Gemini team!

Full leaderboard: kombai.com/benchmark
4 replies 3 reposts 36 likes 8.9K views View on X
Kombai Verified account @Kombaico
Kombai's plugin for Codex is live!

You can now use Kombai's Taste Agent and editable canvas directly inside Codex. No more switching back and forth to the Kombai desktop app.

Here are a few things you can do with the plugin:

- Build standout UIs for your ideas
- Explore creative directions and compare designs side by side on an infinite canvas
- Refine your designs until they're ready to ship
- Generate images and videos for your designs using leading AI models

Try Kombai in Codex and earn up to 350 free credits!

Get started: chatgpt.com/plugins/plugin…
0:51