Twitxy Saved

Harshit Joshi

Verified account This account is verified. Learn more

@harshitj__

CS phd @StanfordNLP, @StanfordOVAL | prev: @MSFTResearch | LLM systems for knowledge access, discovery and curation
Stanford, CA harshitjoshi.com Joined May 2012
429 Following 2.5K Followers 1.1K Posts

Latest posts 5

Pinned post

Harshit Joshi Verified account @harshitj__
Scientific 🔬 progress often starts with an unexpected observation that sparks a new hypothesis.

But AI agents often start with a hypothesis, then look for observations to support it.

We believe agents should actively explore both observations and hypotheses. When observations are only a byproduct of hypothesis testing, we risk missing discoveries we weren’t looking for.

⚡ Meet SHELDON: a scientific reasoning harness that models separate hypothesis and observation spaces and generalizes across medical diagnosis, system-log investigations, ARC-AGI 2, and even autoresearch.

Bazinga!
0:17
4 replies 23 reposts 101 likes 7.5K views View on X
Harshit Joshi Verified account @harshitj__
the only human written review we got for iclr submission says that the presentation is "Clear and easy to follow"

A clearly AI generated review says presentation is "Difficult to follow despite careful reading." :)
Harshit Joshi Verified account @harshitj__
really love this idea of using privileged info to predict scaling laws for weak models!
Suze van Adrichem Verified account @suze_va Nov 3
Can we forecast benchmark accuracy from models that score zero?

We (co-leads @LukeBailey181 @rckpudi) turn a benchmark into a series of easier hinted versions. Jointly modeling scale C and hint level H enables forecasting GSM8K acc (H=0) within 3 pts from 5600x less compute 🧵
View quoted post on X
0 replies 3 reposts 12 likes 919 views View on X
Harshit Joshi Verified account @harshitj__
We should try to reproduce before blindly using AI for discovery! Check out cool findings from George’s latest work.
Shicheng Liu Verified account @ShichengGLiu Oct 22
(1/n) Before trusting AI to make scientific discoveries, can it reproduce existing research - and go one step further to test whether conclusions survive reasonable alternative analytical choices?

Sharing work from this summer with @DbrxMosaicAI, @StanfordNLP, and @StanfordOVAL on using AI agents to reproduce published medical findings and autonomously investigate analytical choices that could change their conclusions.
View quoted post on X
Harshit Joshi Verified account @harshitj__
although suggesting the next message could be useful, it might also limit users thinking and lead everyone to converge on the same analyses, experiments, and results. I believe research thrives on diversity, with everyone trying different things -- really hope we don't lose that
OpenAI Developers Verified account @OpenAIDevs Oct 9
Now in beta: composer predictions in Codex for Pro users.

Codex can now suggest your next message based on your conversation and how you talk to it.

One of the most loved new features we've ever tested internally.
View quoted post on X