Twitxy Saved

Xinming Tu

Verified account This account is verified. Learn more

@TuXinming

PhDing @uwcse, MTS @phylo_bio | Research about Research, Eval on Agentic Science, AI for Genomics | Previously @PKU1898, @msrasia, @genentech
Seattle, WA XinmingTu.cn Joined January 2019
712 Following 2.3K Followers 1.5K Posts

Latest posts 5

Pinned post

Xinming Tu Verified account @TuXinming
Claude was trained on protein sequences, at scale.
We gave 20 language models the start of little-studied bacterial proteins and asked for the next amino acids. Only the latest Claude models predict them. Not GPT, not Gemini, not seven open-weight models.
32 replies 116 reposts 863 likes 131K views View on X
Xinming Tu Verified account @TuXinming
Agree that for superhuman-level science evals, the GT should come from ground-truth experimental data, not a solution written by human.
Nima Alidoust Verified account @nalidoust Nov 9
What makes a biological reasoning task verifiable? Our team @tahoe_ai has spent the last few months on that question and landed on one conviction: for human-difficult tasks with verifiable rewards, the answer key has to come from ground-truth experimental data.

Along the way we built thousands of such environments, which we call TahoeMysts, and we use them to evaluate frontier models (among other things). Every so often a new model produces a big jump, and it is always exciting. Today is one of those days: @claudeai Mythos 5.2!

We will say more about this work in the future. In the meantime, take a look at our results today.
View quoted post on X
1 replies 1 reposts 11 likes 1.3K views View on X
Xinming Tu Verified account @TuXinming
Nice move on these auto-optimizations/auto-research. These improvements should be discussed with the original maintainer. It鈥檚 much more helpful than dropping numbers with some patched code distributed on GitHub.
Jacob Schreiber @jmschreiber91 Oct 29
Time for the biggest drop of the week: auto-optimizations of eleven bioinformatics packages, focusing on read alignment. Each auto-optimization includes: a new unit test suite, the optimizations stacked on top, and a list of issues found during the validation.
View quoted post on X
2 replies 1 reposts 9 likes 2.2K views View on X
Xinming Tu Verified account @TuXinming
So, are there 8 models better than the best human? @ZimingLiu11
Ziming Liu Verified account @ZimingLiu11 Oct 24
Humans can still surpass LLMs in training intuition (in some cases).

Indeed, Kaiming He used his intuition to create ResNet, without producing 1M-token CoT like today's LLMs do.

Introducing Architecture IQ -- Measuring training intuition. Even top LLMs are far from perfect.
View quoted post on X