Twitxy Saved

Tim Althoff

@timalthoff

Jean-Loup Baer Professor @UWCSE seeking to better understand and empower people through data and computation. Recruiting PhD students and Postdocs
Seattle, WA timalthoff.com Joined January 2012
1.8K Following 5K Followers 1.6K Posts

Latest posts 5

Tim Althoff @timalthoff
Our ability to meaningfully evaluate language models is critical, but challenged in many ways.

Check out @kenqgu's work that finally helps us disentangle "reasoning ability" from "parametric/world knowledge", while allowing for scalable, automated evaluation.
True intelligence = reasoning about new information, not memorized facts.

How can we scalably create benchmarks that are completely novel yet have known answers?

Meet SynthWorlds, an eval & data-gen framework to disentangle reasoning and knowledge⬇️🧵

📄arxiv.org/pdf/2510.24427
View quoted post on X
0 replies 1 reposts 7 likes 1.9K views View on X
Tim Althoff @timalthoff
Most complex tasks feature ill- and underdefined goals, at least initially.

Check out @vysrini's paper demonstrating that agents need to do a lot more than follow instructions, incl. refining goals, and balance competing objectives.

Current frontier models fail at these tasks
Excited to share our paper, Substance over Style: Evaluating Proactive Conversational Coaching Agents, accepted to ACL 2025 Main Track!

LLMs do well when given clear context up front.

But what if that context doesn't exist yet?
(1/5)

🔗Read more: arxiv.org/abs/2503.19328
View quoted post on X