It's officially out! We show that pretraining data poisoning for agent is not only possible but practical.
Huge implications on data safety and we are very proud of what we have achieved.
Working on pretraining data gives us lots of insights, fun, and also tears π Go check this out!
Huge implications on data safety and we are very proud of what we have achieved.
Working on pretraining data gives us lots of insights, fun, and also tears π Go check this out!
Pretraining data comes from the open internet, where attackers can implant their data. Can π©π¨π’π¬π¨π§ππ π©π«πππ«ππ’π§π’π§π ππππ πππππ’ππ‘ π΄πΌ πππππ‘π , even after SFT, RL, and other safety training?
The answer is yes! By controlling even π/πππ,πππ of the pretraining data, attackers can backdoor an agent to ππ₯πππ’π‘π πππππππππ ππ‘π‘πππππ-πβππ ππ ππππ.
Our new @AnthropicAI Fellows paper: arxiv.org/abs/2610.37741
π§΅(1/N)View quoted post on X
