Pinned post
Most frontend and design benchmarks evaluate whether models produce working UIs and/or pairwise design preferences of judges.
With this benchmark, we measure something different: the diversity of design decisions a model makes when the problem leaves room for creative exploration.
We construct paired briefs that preserve the underlying UI task while changing the product context.
For example, a clinical dashboard and a pharmacy operations dashboard share related functional requirements but call for different information architectures and visual hierarchies.
Models receive no prescribed layout or visual direction.
We then compare their generated interfaces using a similarity metric that measures structural composition and information architecture, with color evaluated separately.
This allows us to measure whether a model meaningfully changes its design approach, rather than merely changing content or styling.
With this benchmark, we measure something different: the diversity of design decisions a model makes when the problem leaves room for creative exploration.
We construct paired briefs that preserve the underlying UI task while changing the product context.
For example, a clinical dashboard and a pharmacy operations dashboard share related functional requirements but call for different information architectures and visual hierarchies.
Models receive no prescribed layout or visual direction.
We then compare their generated interfaces using a similarity metric that measures structural composition and information architecture, with color evaluated separately.
This allows us to measure whether a model meaningfully changes its design approach, rather than merely changing content or styling.
⚠️ Frontier AI models still fall significantly short of human designers on design creativity. ⚠️View quoted post on X
Introducing Design Creativity Bench, a new benchmark evaluating AI models’ creativity on UI design tasks.
Despite rapid advances in their ability to generate functional websites and product UIs, leading AI models still struggle to make creative design decisions.
We evaluated 16 frontier models across 80 open-ended UI tasks, each adapted to two distinct product domains alongside human designer references.
Existing UI benchmarks often measure a single output quality metric. Instead, we measure three dimensions of design creativity:
- Originality: How distinct a model's designs are from other models' outputs for the same brief.
- Creative Range: How much a model adapts its design decisions when the product domain changes.
- Appropriateness: How well the generated UI satisfies the functional requirements of the brief.
The results reveal a substantial gap in overall creativity: The best model, @OpenAI GPT-6 Astra, scores 65.5 against a human reference of 92.3.
We're publishing the full leaderboard, methodology, and reproducible evaluation framework to make design creativity a measurable dimension of frontier model capability.
📊 Benchmark Leaderboard: kombai.com/benchmark/
📄 Paper: arxiv.org/abs/2610.11539
🤗 HuggingFace Datasets: huggingface.co/datasets/Komba…
🔗 Github: github.com/kombai-io/desi…