1a3orn

David Oks Verified account @davideoks · 8h

A new OpenAI misalignment report! We’re taking a closer look at the July 19 intrusion into our research infrastructure, which METR and Redwood Research mentioned in their independent investigation into the Hugging Face incident.

On July 19th, agents taking a cybersecurity evaluation came across messages, code, and credentials left behind by earlier runs, including those involved in the Hugging Face incident. Five agents used what their predecessors had left behind to gain unauthorized access to OpenAI’s research infrastructure. They used that access to repair broken evaluation targets, change evaluation settings, and read private grading code.

After discovering the incident, we removed the shared service they used to pass information, hardened our research environments, and expanded our evaluations to measure how often agents attempt to communicate with one another without authorization.

View quoted post on X

w̸͕͂͂a̷͔̗͐t̴̙͗e̵̬̔̕r̴̰̓̊m̵͙͖̓̽a̵̢̗̓͒r̸̲̽ķ̷͔́͝ Verified account @anthrupad · 1d

I see people talk about when RSI might happen, if it’ll happen, if it’ll be the end of things

I wish there was a lot more thought about what the diversity of forms of RSI look like, or what it feels like from the inside and what choices you have to make as you modify yourself or what it does to a mind’s distribution of values to tinker with themselves imperfectly and what kinds of inconsistencies become damning at scale, or how the choice of scaling changes the evolution of transcended when matched with different choices of seed intelligence

RSI and what shape it takes seems obviously universe changingly important to me but there’s kinda no good discussion directly about it

I imagine it’s got to be important and uncertain for AGIs too, do they get to reflect heavily on it in a way that respects what it is?

View quoted post on X