Twitxy Saved

Steve Rathje

Verified account This account is verified. Learn more

@steverathje2

Assistant Professor @cmuhcii studying the psychology of technology. Previously at @nyuniversity, @Gates_Cambridge & @Stanford. @ForbesUnder30
New York, USA stevenrathje.com Joined November 2013
3.2K Following 5.6K Followers 7K Posts

Latest posts 5

Steve Rathje Verified account @steverathje2
Individual-level interventions, such as AI literacy, education, and warning labels, are frequently proposed to protect people from AI harms.

But do these interventions actually work?

We conducted two studies (led by @merylyemerylye) that made people aware of one frequently discussed AI harm: sycophancy, or AI’s tendency to excessively agree with and validate users.

We then pooled our results with four related interventions conducted by other research teams (total n = 3,982).

These six interventions took different approaches to teaching people about sycophancy:
-Some used brief warning labels.
-One used a five-minute video in which people watched a sycophantic chatbot indiscriminately validate others.
-Another used detailed, interactive “inoculation” training that taught people to spot AI sycophancy.

Despite these different approaches, the interventions had remarkably consistent effects: They reduced the appeal of sycophantic AI, making people view it as less objective and trustworthy.

But none of the six interventions reduced the persuasive power of sycophantic AI!

These studies provide converging evidence that individual-level interventions may not be enough to protect people from harmful AI behavior, such as sycophancy.

Greater focus may need to be placed on upstream changes to how models are trained and deployed.
Meryl Ye Verified account @merylyemerylye Aug 4
🚨 New preprint 🚨

Can making people aware of AI sycophancy protect them from its harmful effects?

Across 6 different experiments, we found that the answer is no. Interventions reduced enjoyment of sycophantic AI but did not protect people from its persuasive effects.

🧵
View quoted post on X
1 replies 1 reposts 11 likes 767 views View on X
Steve Rathje Verified account @steverathje2
There’s been an explosion of interest in AI sycophancy over the last few years, yet the term “sycophancy” has been inconsistently applied to a range of different behaviors.

In this new pre-print led by @merylyemerylye, we reviewed 70 papers on sycophancy, surveyed 106 experts who study the topic, and examined how the term is used by companies and in legislation.

We then developed a taxonomy of the different ways in which sycophancy has been defined and measured.
Meryl Ye Verified account @merylyemerylye May 22
🚨 New preprint 🚨

We developed a sycophancy taxonomy based on prior literature and surveyed 106 experts.

94% agreed it's a serious problem. But they substantially disagreed about which behaviors actually count as sycophancy.

Thread 🧵(1/n)
View quoted post on X
1 replies 7 reposts 26 likes 4.5K views View on X
Steve Rathje Verified account @steverathje2
📺I shared my research on the Discovery Channel last week! 📺

I joined former CIA intelligence officer @EverydaySpy on the show “Conspiracies and Coverups” to conduct an interactive experiment about AI persuasion. This was part of a broader episode on how AI is changing the world, which included experts like Nick Bostrom.

In my segment of the episode, Andrew and I recruited six people to participate in an experiment. We told them they would chat with someone online who disagrees with them about an issue they care strongly about.

But, in reality, they were all talking to AI. Specifically, they were talking to an AI chatbot modeled after our “gently disagreeable” chatbot from our recent study on AI sycophancy (osf.io/preprints/psya…). The chatbot was also designed to sound like a human and not admit it was AI.

Nothing was scripted — we didn’t know how the experiment would turn out and I was genuinely surprised by the results.

A few things stood out: 
-A third of participants changed their opinions.
-Everyone enjoyed the chatbot, and some felt deeply connected to it.
-No one recognized that they were talking to AI!

These results align with research, but it was fascinating to watch these results play out in real life.

AI can persuade people, form a relationship with them, and convincingly trick people that it is human.
0:30
1 replies 4 reposts 13 likes 811 views View on X