Twitxy Saved

Eyon Jang

Verified account This account is verified. Learn more

@eyonjang

alignment research @scale_ai | prev: MATS 8
San Francisco, CA eyonjang.me Joined December 2012
398 Following 282 Followers 26 Posts

Latest posts 4

Eyon Jang Verified account @eyonjang
please consider applying if you're interested in safety/alignment research! my team will be looking for 2 interns to join.
Scale Verified account @scale_AI Oct 27
From the beginning, Scale was built by people who love solving hard problems. That same ambition is driving what we build next.

Our ML Research Internship Program is accepting applications in SF and NYC. Work on frontier AI research alongside @ScaleAILabs researchers, publish your work, and help build the next generation of AI.

Apply now. scale.com/careers/univer…
View quoted post on X
8 replies 7 reposts 244 likes 14K views View on X
Eyon Jang Verified account @eyonjang
I’m mentoring with @pivotal_org this winter.

Work with me and @ihsgnef on alignment auditing. We’re looking for rigorous empiricists who understand the alignment research landscape and switch easily between fuzzy conceptual thinking and engineering work.

Apply by 1 Nov!
Pivotal Research Verified account @pivotal_org Oct 9
Applications are open for Pivotal’s Q1 2027 Research Fellowship 💥

Work with mentors from @AnthropicAI , @GoogleDeepMind, @AISecurityInst, @ApolloResearch, and more.

📍 15 weeks in London (18th Jan - 30th Apr).
💸 £15-18k stipend, travel and housing support

Apply by Nov 1: apply.pivo.tl ↓
View quoted post on X
Eyon Jang Verified account @eyonjang
Can LLMs learn to resist RL training?

We empirically study exploration hacking: models controlling behavior during RL to prevent unwanted capabilities from being reinforced.

Joint work with @GoogleDeepMind and @MATSprogram.

More in the thread below 👇
Joschka Braun Verified account @BraunJoschka May 4
RL assumes that LLMs explore well during training. What if they choose not to?

In our new ICML paper with @GoogleDeepMind, we train LLMs that strategically resist RL capability elicitation by under-exploring.

We study this threat model, called exploration hacking.
View quoted post on X
1 replies 1 reposts 10 likes 766 views View on X
Eyon Jang Verified account @eyonjang
Excited to share our conceptual framework for exploration hacking. Check it out!
Joschka Braun Verified account @BraunJoschka Feb 12
1/ What if AI models could resist or influence their own RL training—by strategically choosing what not to explore? Ahead of our upcoming empirical paper, we formalize and decompose "exploration hacking." New conceptual framework post with @eyonjang & @DamonFalck 🧵👇
View quoted post on X