!
Many AI control protocols block actions a monitor flags as suspicious. In the near-term, this might be the main pillar mitigating AI loss-of-control. In a new post, I argue effective continual learning would plausibly make your blocking monitors nearly useless.View quoted post on X
Continual learning mechanisms like online training teach agents to be more useful based on experience from their deployment. Being blocked by a monitor interferes with task success, so a continually-learning agent would learn how to evade blocking monitors, even if it starts out benign.
You'd probably notice this issue, but I argue it’d be costly to fix: to the extent that evasion is sometimes hard to tell apart from legitimate learning, you’d have to make a decision between giving up on a bunch of legitimate learning or accepting your agent learning how to evade blocking monitors. I discuss mitigations, including: improving monitors, making protocols interfere with usefulness less, and using counterfactual rewards.
lesswrong.com/posts/QnDqGbKe…
(This post was mostly written ~3 months ago, prior to all of the recent incidents during training and evaluations.)