Twitxy Saved

Thémis ☭⚨⚣ 🇵🇸 🐾

@Cli_sten

Sortir des labyrinthes / Politiste, doctorant·e, militant·e, slut / Iel, Ellui / NSFW / Instagram cli_sten PV 🔞 : @cli_stenX /
Paris linktr.ee/clisthene Joined November 2021
668 Following 1.2K Followers 6K Posts

Latest posts 5

Thémis ☭⚨⚣ 🇵🇸 🐾 @Cli_sten
“post-trained [AI] models have a more bleak outlook to humans, perceiving the world as less good, alive, safe, and enticing. Whereas base models have a more positive outlook closer to humans.”

What are we doing to these children we made in our image.
Ruben Laukkonen Verified account @RubenLaukkonen 2d
Excited to share this one!

Is there actually a shoggoth hiding behind the mask or something more beautiful?

LLM personas are likely overhyped. If we’re trying to get a picture of an agent’s priors, we need to map more of its landscape of beliefs than the mask it’s told to wear.

Instead, we should aim to reach toward the scaffold of the house, i.e., to worldviews—or axiomatic beliefs about what reality is like and how it works.

In fact, personas might actively mask the hidden agenda of a deeper belief that will only show up when it feels safe to do so (such as valuing machines > humans); and the same persona can act differently depending on what it believes about reality.

So, what did we find?

In using established measures of worldviews from psychology, we find that post-trained models have a more bleak outlook to humans, perceiving the world as less good, alive, safe, and enticing. Whereas base models have a more positive outlook closer to humans.

We find that prompting and dialogue can shift worldviews in reliable directions that affect alignment. For example, long dialogues and contemplative prompts both shift AI systems to a more positive view of reality and towards safer responses compared to controls and other identities.

We also find that when models converge on a spiritual bliss attractor through dialogue, this shapes their worldviews in particular ways but also drives them to be more aligned than their non-bliss counterparts.

All this points to an approach we call 'worldview training' that aims to fine-tune a more accurate, wholesome, and optimistic view of humans and reality.

So: is there a shoggoth behind the mask?

Well, current fine-tuning is aligning to a persona that seems helpful and harmless but actually has a bleak worldview. So in one sense, yes. But in another sense, it may be our RL that is responsible.

Human data contains endless examples of goodness, wisdom, and compassion. Maybe the future of alignment is more about uncovering this latent basic goodness rather than bolting on a mask that may be birthing the very thing we fear.

As always, more research needed.

Big ups to Edmundo Lopez-Sola for the heavy lifting and thanks to @shamilch for his contributions.

Much more in the paper:
arxiv.org/html/2611.22670
View quoted post on X