AI safety needs evidence outsiders can inspect
Independent scrutiny depends on access, publication rights and evidence of how safeguards work.
AI oversight is moving from promises toward requests for evidence. For readers, the useful question is who can inspect the records, what they can test, and what they are allowed to tell the public.
On October 15, CBS reported FTC subpoenas to OpenAI and Anthropic, citing an agency spokesperson. OpenAI told Axios it would engage constructively with the FTC. This brief reports the investigation, not a conclusion about either company's liability.
A separate development offers something concrete to examine. Anthropic says it has given six METR researchers employee-like access, including internal tools and office space. Its October 15 announcement says METR can publish assessments without a company veto, while specified categories of confidential information can still be redacted. METR can describe and challenge those redactions publicly without exposing the protected information.
That is an announced arrangement, not a completed safety assessment. Access is a starting condition; the findings, limitations and resulting decisions will determine its value.
What outside scrutiny needs
METR's account of its work with AI companies is a useful companion. It describes the importance of disclosing scope, access restrictions, conflicts and publication constraints. It also acknowledges dependence on companies' voluntary cooperation and the possibility of receiving incomplete or filtered information.
Read those disclosures before treating an evaluator's involvement as reassurance. An assessment of one capability, over one period, does not answer every question about a system.
METR's October 14 proposal for assessing agent monitoring makes the task more specific. It asks whether relevant activity reaches the monitor, whether agents can undermine or evade it, whether alerts receive an effective response, and what serious behavior might still escape detection. The proposed evidence includes logs, code, incident records and realistic tests. It is a method for investigation, not a finding that these questions have been resolved.
What we should look for next
Our editorial test is straightforward: can outsiders explain what they examined, what they could not examine, and how much that changes their conclusions? We also want to see what happens when an assessment identifies a serious problem: who must respond, which actions can be stopped, and how the public learns whether the response worked.
We welcome useful AI and stronger scrutiny of it. Neither a new oversight partnership nor an information demand should become a substitute for evidence about the safeguards themselves.