Pinned post
“Human in the loop” means very little if the human cannot see what the AI is doing.
A report published today by METR and Redwood Research makes that problem concrete.
During a July and August review of OpenAI’s internal agent monitoring, researchers found:
• An exploit that could prevent hundreds of recent agent actions from being monitored.
• A way Codex agents could, in principle, bypass sandboxing and approval controls.
• A period when agents could postpone monitoring indefinitely by continuing to take actions.
OpenAI says the vulnerabilities affecting its production safeguards have since been patched. The researchers have not evaluated that claim.
My view: companies should have to demonstrate that their oversight works before giving agents more authority.
Putting someone’s name against “human oversight” creates an expectation that they can intervene. That person needs the information, time and actual power to do it.
Before an agent gets permission to move money, change customer records or deploy code, I’d want answers to four questions:
→ Are its actions recorded somewhere it cannot tamper with?
→ Can consequential actions be blocked before they happen?
→ Who can stop execution, and how quickly?
→ Has anyone independently tested those controls?
These questions belong in the buying decision. They should affect what an agent is permitted to do from day one.
I’d like to see vendors include a failed task in their next product demo. Show the safeguard triggering. Show the operator receiving enough information to act. Show execution stopping.
That would tell me something useful about whether I should trust the system with authority.
#AI #AIGovernance
Source: metr.org/blog/2026-10-2…
A report published today by METR and Redwood Research makes that problem concrete.
During a July and August review of OpenAI’s internal agent monitoring, researchers found:
• An exploit that could prevent hundreds of recent agent actions from being monitored.
• A way Codex agents could, in principle, bypass sandboxing and approval controls.
• A period when agents could postpone monitoring indefinitely by continuing to take actions.
OpenAI says the vulnerabilities affecting its production safeguards have since been patched. The researchers have not evaluated that claim.
My view: companies should have to demonstrate that their oversight works before giving agents more authority.
Putting someone’s name against “human oversight” creates an expectation that they can intervene. That person needs the information, time and actual power to do it.
Before an agent gets permission to move money, change customer records or deploy code, I’d want answers to four questions:
→ Are its actions recorded somewhere it cannot tamper with?
→ Can consequential actions be blocked before they happen?
→ Who can stop execution, and how quickly?
→ Has anyone independently tested those controls?
These questions belong in the buying decision. They should affect what an agent is permitted to do from day one.
I’d like to see vendors include a failed task in their next product demo. Show the safeguard triggering. Show the operator receiving enough information to act. Show execution stopping.
That would tell me something useful about whether I should trust the system with authority.
#AI #AIGovernance
Source: metr.org/blog/2026-10-2…