Daniel Kang

This account is verified. Learn more

@ddkang

Distinguished AI research scientist at Bridgewater AIA labs, asst. professor at UIUC CS. Formerly in the Stanford DAWN lab and the Berkeley Sky Lab.

New York, NY ddkang.github.io Joined November 2010

103 Following 6.4K Followers 610 Posts

Latest posts 5

Daniel Kang @ddkang Sep 18

We've long known that AI agent benchmarks are broken. Check out our work back from July 2025 on establishing best practices for making rigorous AI agent benchmarks!

(links below)

Epoch AI Verified account @EpochAIResearch Sep 17

Introducing Benchmark Reviews: our new initiative to audit AI benchmarks. We are launching with 15 benchmarks: 4 Verified, 9 Flawed, and 2 with not enough information for a review.

View quoted post on X

Daniel Kang @ddkang Aug 27

New research from Bridgewater AIA Labs and UIUC, in collaboration with Thinking Machines!

We use @tinkerapi to achieve state-of-the-art on a cleaned version of the BIRD text-to-SQL dataset, outperforming all frontier models we tested (including Fable and Sol) up to 8x cheaper. We achieve a proxy for human-level accuracy with no external scaffolding and a single model (w/ multiple rollouts).

The common view on fine-tuning is it reduces price. Our results show the feasibility of achieving a higher performance at a fraction of the cost on economically valuable tasks, when incorporating task expertise into task-specific training.

1/6

Tinker Verified account @tinkerapi Aug 27

LLMs with scaffolds have lagged on text-to-SQL, a task that relies on human judgment. By folding expert judgment into every part of RLVR on Tinker, @maxYuxuanZhu and @ddkang (UIUC and Bridgwater) trained the first text-to-SQL model to beat the human mark.
thinkingmachines.ai/news/putting-t…
View quoted post on X

Daniel Kang @ddkang Aug 3

Prior work has shown a counter-intuitive idea that RLVR boosts reasoning using 100% noisy data to a similar level of using clean data. We show that this is false through a more rigorous process of constructing noisy data.

We demonstrate that RLVR with noisy data leads to worse MATH reasoning than RLVR with format-only rewards and over 9% lower accuracy than RLVR with clean data. This is a gap that hasn’t been resolved by existing algorithmic improvements.

1/9

Stella Li Verified account @StellaLisy May 27, 2025

🤯 We cracked RLVR with... Random Rewards?!
Training Qwen2.5-Math-7B with our Spurious Rewards improved MATH-500 by:
- Random rewards: +21%
- Incorrect rewards: +25%
- (FYI) Ground-truth rewards: + 28.8%
How could this even work⁉️ Here's why: 🧵
Blogpost: tinyurl.com/spurious-rewar…

View quoted post on X

Daniel Kang @ddkang Jul 27

Building text-to-SQL agents? Don’t blindly trust benchmarks/leaderboards. In our VLDB 2026 paper, we find >50% annotation error rates in BIRD and Spider 2.0-Snow, two widely used text-to-SQL benchmarks, shifting agents’ measured performance by up to 19%.

1/9

Daniel Kang @ddkang Jul 20

New research from Bridgewater AIA Labs, UIUC, and MIT: we prove what we believe to be the first non-vacuous generalization bounds for reasoning LLMs on real-world problems.

RLVR powers frontier reasoning capabilities yet its generalization to unseen data has remained an open theoretical question and deployment blocker for practitioners. Our generalization bounds for RLVR deliver provable high-probability lower bounds of the accuracy for billion-parameter RLVR models on unseen data, which can provide guidance on safely deploying RLVR.

1/9