LLMs' social sycophancy can make us less willing to repair past wrongs. We auto create preference data that penalizes responses harmful to latent stakeholders. Tuning on such yields replies closer to human views. w/great co-authors incl. @HatgisKessell @natashajaques @chengmyra1
🧵Thread (1/9): People increasingly turn to AI for personal advice, and LLMs tell them what they want to hear, endorsing the user far more often than humans do.View quoted post on X
The problem: people become less willing to repair their relationships after conflicts.
We propose PlurPO, a method to construct a pluralistic preference dataset to post-train models to mitigate this social sycophancy.