Twitxy Saved

Minglai Yang

Verified account This account is verified. Learn more

@Yminglai

Research Scientist @ScaleAILabs|Prev @labclu @thukeg, working on agents, rlenv, usersim
San Francisco, CA ymingl.com Joined May 2024
208 Following 488 Followers 252 Posts

Latest posts 5

Minglai Yang Verified account @Yminglai
I think much of the value in AI data will come from expert level data, the ability to source the right experts, and the research capability to understand where their judgment can actually improve a model. Synthetic data is relatively easier to scale. What I find more interesting is how to incorporate human intelligence into tasks that models still struggle with. There are a lot of unresolved questions here, including what kind of expertise we need and how to capture what an expert knows beyond the final answer.

One complication is that difficulty looks different for humans and models. Some tasks require years of training for a person but may already be relatively easy for a model. Others feel simple or intuitive to a human, yet models struggle with them. How much effort a human spends on a task does not necessarily tell us how valuable that task is for training.

This makes “expert data” a more complicated category than it first appears. Hiring people with strong credentials is only part of the problem. We also need to understand which parts of their judgment are missing from the model, whether those judgments can be expressed in useful data, and whether learning from them carries over to other situations.

I suspect a lot of the opportunity sits in those questions. Access to experts matters, but so does having the research capability to figure out what to ask of them. Even with an excellent group of people, it is possible to collect expensive, sophisticated work that teaches a model very little.

The question I keep coming back to is: which kinds of human intelligence are still difficult for models to acquire, and how do we make those learnable?
Jinyan Su Verified account @SuJinyan6 2d
What I saw of the AI data business while job hunting for post-training roles, and why it keeps booming even though it seems to have no moat. x.com/i/article/2121…
View quoted post on X
Minglai Yang Verified account @Yminglai
How to control RSI is a critical problem. Amazing work by @KaiwenZhou9
Kaiwen Zhou Verified account @KaiwenZhou9 5d
🚀 Sharing our new paper on RSI Safety

AI labs are working toward RSI. However, we need to know that loop is safe before it closes.

Introducing RSI Control from @ScaleAILabs: What kind of hidden goals AI may pursue when training the successor? Can AI models perform covert sabotage in the self-Improvement process? And can a monitor detect the sabotage?

We tested whether frontier models can complete a secret side task while working on tasks across the model training pipeline, from data curation to RL. The side tasks include changing the successor's capabilities, changing its alignment, and injecting a goal into it.

🔍 What we found:
1. Every model could do it. 36% of 374 attack runs complete the side task while keeping the main task performance. This includes making the successor a better saboteur.
2. Monitors can’t catch some of the attacks. At a 2% false-positive rate, transcript monitors catch only 56–81% of them.
3. Give agents a short list of evasion strategies and the monitors’ true-positive rate drops to 16–28%.
Reading the agent's CoT catches most attacks, but not all.

Strong oversight has to be in place before the RSI loop closes.

📑Paper: arxiv.org/abs/2610.27735
More in 🧵👇
View quoted post on X
1 replies 0 reposts 11 likes 589 views View on X
Minglai Yang Verified account @Yminglai
Proud of the team behind this launch. I’ve been fortunate to co-lead the research behind these RL environments: 3,300+ tasks designed by domain experts around real-world enterprise workflows, spanning tool and computer use. Building frontier agents? Reach out 😎
Francis deSouza Verified account @fdesouza Nov 5
Today we're publishing a major RL environments update: 3,300+ tasks across tool use and computer use, already running with customers. Some tasks in our newest RL environments take a human team 3 weeks and 500+ tool calls.

scale.com/rlenvironments
View quoted post on X