jacob tsimerman (@Jacob_Tsimerman)

Geoffrey’s posts are always worth reading, but this one is a must-read. It lays out a really clear high-level hierarchy of safety levels, separated by which standard of rigor they adhere to, from “slapdash empirics” to “unconditional proofs”. The AI industry is unfortunately currently operating at the bottom level.

I think it is really important for the public to understand that it is within society’s power demand the higher levels! Not everything needs formal. verification and mathematical proofs, but
1. It is reasonable to demand that for the riskiest technologies, and
2. There are intermediate standards of rigor that are way better than what we have now.

Geoffrey Irving Verified account @geoffreyirving Oct 22

Lots of conversation around AI risks is about the gap between the unserious of AI labs' methods and how more rigorous industries work: airplanes, nuclear power, AI hardware. It would help a ton to lift AI to that level, but it is unclear even that level would be enough. 🧵

View quoted post on X

jacob tsimerman @Jacob_Tsimerman Oct 22

I’m honoured to be on the advisory board of the new Alignment Journal, and we open for submissions today! The field of AI safety needs more canonical, high-quality spaces for exposition (despite the wonderful efforts of many, many people, including Victoria Krakovna who is on the advisory board as well!) and I think this could be a great addition.

Please take a look, and if you're doing research on AI safety then (Thank You! And) consider submitting!

AlignmentJournal Verified account @AlignmentJrnl Oct 22

The Alignment Journal publishes research on the scientific foundations of understanding, predicting, and steering artificial intelligent systems. As of today, submissions are open to everyone.

What we do differently:
- We pay reviewers, with bonuses for speed and insight.
- Review discussion is confidential, but every accepted paper is published with a “review summary” to distill the insights generated.
- Each paper is judged against our acceptance bar, not against other submissions.
- Papers under review are listed publicly, and researchers can nominate themselves to review, so each paper can reach the reviewers who know the topic best.

We seek work that builds theoretical or otherwise principled analyses of agency, robustness, incentives, interpretability, generalization, and long-term behavior, and whose significance survives the deletion of any particular model, dataset or benchmark. Empirical investigations of such ideas are welcome, and negative results are encouraged.

We are especially interested in approaches that could apply to future superintelligent systems that interact and self-improve.

You can submit your preprint in any format. Submissions are rolling and open now on the website.

If you are instead interested in reviewing or serving as an action editor (basically: an area chair), create an account on our website.

Links to submit, our scope and the reasoning behind the Journal are in the reply ⬇️

View quoted post on X

jacob tsimerman @Jacob_Tsimerman Oct 15

I’m excited to announce that I will be working at the Mathematical AI Safety Institute (MAISI) full-time as its founding Scientific Director! This means I will not be joining OpenAI as an employee, though I will still be collaborating with them, and I’m happy to announce that OpenAI will be supporting MAISI. They have offered us $10MM in API credits, and access to their latest models for our research use. Moreover, they will be participating in a multi-lab AI safety workshop we are organizing for next year with other frontier AI labs.

As an independent research institute, we are also in conversations with other labs and AI safety orgs about how MAISI can collaborate with them and contribute quickly to AI safety research across the industry. As we’ve seen over the past few weeks, the need for action on AI safety is extremely urgent. Research is an important part of that, and mathematicians have an important role to play!

jacob tsimerman @Jacob_Tsimerman Sep 8

I'm incredibly excited to announce the founding of the Mathematical AI safety Institute (MAISI) maisi.org. AI safety needs more foundational theoretical development, and mathematicians have the skills and the mindset to help! MAISI is an independent institute with visiting positions ranging from 1 semester to 2 years. Our goal is to get mathematicians up to speed and working on research directions in AI safety as quickly as possible. There is important work to be done, and there is real progress to be made. YOU can help!

Applications are open now! MAISI is aiming to hire 10-30 mathematicians to join us in the Bay Area by January, and scale up to 30-100 for September 2027. If you're a mathematician interested in channeling your skills toward the most important problem of our time, please apply today! maisi.org/apply

maisi.org Mathematical AI Safety Institute A mathematical research institute for AI safety.

jacob tsimerman @Jacob_Tsimerman Aug 3

A couple of websites

I want to share a couple websites aimed specifically at mathematicians regarding AI-safety and AI-risk .

The first one I made myself: mathforaisafety.org. This is designed to be a very high level and accessible resource describing some research agendas, collections of problems, and organizations in the field of AI-safety. I hope that for mathematicians considering directing some of their effort in this direction, this eliminates some of the friction. I plan to expand it quite a bit, but I hope it's already useful! I'd be happy and grateful to hear (constructive) feedback on it :-)

1/2

maisi.org Mathematical AI Safety Institute A mathematical research institute for AI safety.

jacob tsimerman

Latest posts 5