Twitxy Saved

Sohyeon Kim

Verified account This account is verified. Learn more

@ohmyksh

Retrieval and memory for open-ended discovery
Stanford, CA ohmyksh.github.io Joined February 2024
174 Following 903 Followers 141 Posts

Latest posts 5

Pinned post

Sohyeon Kim Verified account @ohmyksh
AI can increasingly make progress on open problems like Navier-Stokes. Yet defining a new problem still relies on human research taste.

Introducing ScholarCatalyst, a benchmark built from AI researchers’ firsthand accounts of what inspired their work.
11 replies 66 reposts 301 likes 66K views View on X
Sohyeon Kim Verified account @ohmyksh
Self-improving retrievers at search time by @dianetc_

We’ve mostly studied training retrievers with offline relevance judgments. But what happens in a realistic, unlabeled setting, where no relevance judgments exist?

We can continuously teach retrievers where to look using a reasoning model’s preferences. This is especially exciting for agent memory (and agent swarms!), where search experience can accumulate and benefit future agents.

Lots of exciting things ahead!
Diane Verified account @dianetc_ Nov 6
Is there a way to scale a reasoning model’s judgment for searching a massive corpus?

We introduce the Amortized Reasoning Index (ARI), a new kind of retriever that updates its weights during search, using gradients derived at inference time from the ranking preferences of a reasoning model.

Every session, current agents spend lots of reasoning deciding which search results matter, and then throw all of that judgment away upon compaction. ARI is our attempt to “amortize” that judgment, so that reasoning spent on past queries keeps improving search on future ones.

With no relevance labels, ARI takes a 400M cross-encoder that starts out weak and grows its Recall@100 on held-out queries by up to 7×, ending above Qwen3-Embedding-8B on various oblique/reasoning-intensive search tasks.

Read our blogpost: dianetc.com/musings/ari/

Preliminary results and discussions are below!
View quoted post on X
0 replies 5 reposts 59 likes 3.4K views View on X
Sohyeon Kim Verified account @ohmyksh
Really enjoyed reading this early draft and thinking more about research taste. Even as AI tackles expert-level problems, we sense a hard-to-define gap between AI and us, often called taste.

I think “taste” takes different forms across research decisions, and we need to study these forms more granularly and develop ways to characterize and evaluate each. Our paper ScholarCatalyst focuses on taste during problem *formation*, collecting previously undocumented firsthand data from researchers.

Taste also shapes how completed papers are *evaluated*, which we can study through existing academic judgments. For an interesting study of this, check out @seungonekim's post below!
Seungone Kim Verified account @seungonekim Oct 6
As AI agents take on more of the research process, we want to understand how its "research taste" differ from ours.

In a pilot study along with @ShayneRedford, we had AI agents act as area chairs and rank all ICML 2026 papers.

We found notable differences between the two!
View quoted post on X
1 replies 0 reposts 9 likes 1.1K views View on X
Sohyeon Kim Verified account @ohmyksh
For Seoulmaxxing @ ICML!

(New Updates)
- Naver Map: naver.me/GlRizXuo
- Vegetarian-Friendly Places: naver.me/GqfuPir8

Check out the website for the latest updates and more details!

ohmyksh.github.io/icml2026/index…
Sohyeon Kim Verified account @ohmyksh Jun 26
If you’re coming to #ICML2026 🇰🇷, this Seoul Guide might be All You Need!

I’ve lived in Seoul for 20+ years and curated this all-in-one guide: Top 5 must-visits, 100+ recs with Google Maps, practical local tips, and more.

Check it out: ohmyksh.github.io/icml2026/index…

DMs are open and happy to give more details!
View quoted post on X