Twitxy Saved

Alex Spangher

Verified account This account is verified. Learn more

@AlexanderSpangh

Post doc at @StanfordHAI and SAIL, with Diyi, Sanmi, Dan Ho. Studies human rewards in creative tasks. PhD @usc, @nytimes journalist, @JuilliardSchool musician
Manhattan, NY alexander-spangher.com Joined February 2012
404 Following 1.7K Followers 601 Posts

Latest posts 5

Alex Spangher Verified account @AlexanderSpangh
@HJCH0 and team are really awesome researchers and this report is amazing!! Looking forward to lots of great work in AI4Science coming out of @intology
Intology Verified account @intology 6d
Locus sets the frontier for automating molecular design

Our automated research system Locus develops scientific AI models that outperform SOTA expert solutions

Unlocking the potential of scientific AI requires leveraging proprietary data. Locus achieves this autonomously 🧵👇

--

We test Locus across a number of tasks related to molecular scoring and design: training molecular property prediction models for a proprietary task, fine-tuning Boltz-2 to predict small-molecule potency, and building antibody design workflows around BoltzGen, Boltz-2, ESMC, and ESMFold2.

Each of these settings are real-world AI for science use cases, where tasks are often out-of-distribution for existing foundation models and limited in data, while featuring expensive and noisy evaluators. Compared to state-of-the-art baselines and LLM agents set to optimize these tasks, Locus achieves higher performance. Additionally, Locus generalizes better despite limited data, finds more sample efficient solutions, and explores a higher diversity of solutions.
View quoted post on X
Alex Spangher Verified account @AlexanderSpangh
🚨New work: A SOTA prompt optimization framework + a novel unsupervised measure of prompt performance.

Written rubrics and constitutions are crucially important for AI improvement, safety following, pluralism and adaptation.

But some of the things we might want a rubric to do are difficult to put into words; how would you even describe “comic voice”?
1 replies 2 reposts 29 likes 1.8K views View on X
Alex Spangher Verified account @AlexanderSpangh
So excited for this research! I've always been annoyed at the supremacy of verified rewards, and now we can finally show it.

In this work, we develop ways of measuring: how much of a reward signal is verifiable, how much can we put into words, and how much is shared but tacit, or impossible to articulate?
4 replies 43 reposts 276 likes 28K views View on X
Alex Spangher Verified account @AlexanderSpangh
🚨New work: A SOTA prompt optimization framework + a novel unsupervised measure of prompt performance.

Written rubrics and constitutions are crucially important for AI task improvement, safety following, pluralism and adaptation.

But some of the things we might want a rubric to do are difficult to put into words; how would you even describe “comic voice”? I’m human, albeit not that funny, and I'm totally not sure what that means.

What kinds of written constructs can you expect your judge to do well on and how good can you expect it to be?