Twitxy Saved

Shi Feng

Verified account This account is verified. Learn more

@ihsgnef

Joined October 2013
585 Following 1.1K Followers 502 Posts

Latest posts 5

Shi Feng Verified account @ihsgnef
Building Comparative Motivation Profiles with Symmetric Belief Editing will appear at NeurIPS.

We refined the methodology and messaging of our performative misalignment work. We are excited to extend this to white-box auditing.

by @David00845018 @1700x0 @MinTaywon @JinghuaO via @MATSprogram

1/7
Shi Feng Verified account @ihsgnef Mar 18
New post: Sycophancy Towards Researchers Drives Performative Misalignment

We found no clear evidence that scheming is more valid than sycophancy to explain alignment faking. 馃У
View quoted post on X
1 replies 5 reposts 27 likes 1.7K views View on X
Shi Feng Verified account @ihsgnef
Covert influence between LMs

LMs consume each other鈥檚 outputs in training and evals. A sender may transfer payloads via carriers that look innocuous to humans.

This occurs across supervised fine tuning, on-policy distillation, and in-context learning.
1 replies 1 reposts 19 likes 765 views View on X
Shi Feng Verified account @ihsgnef
Tim and Bronson provided valid and helpful critiques about our empirics, notably not testing on Opus 3 which alignment fakes "for real". We'd love to try that but don't have SDF or steering access.

The meta point of this exercise, however, is that the logic cuts both ways. 1/6
Tim Hua 馃 Verified account @Tim_Hua_ Mar 19
I think this post is misleading and strong downvoted it on lesswrong. My main critique is that all of the experiments are done on Llama 3.1 70B, a model that doesn't really even alignment fake to begin with.
View quoted post on X