Anthropic lets METR inside. Publication is the real test.

An outside safety review is only useful if the reviewer can see what matters—and tell others what it found. For a business deciding whether to trust powerful AI in its operations, a reassuring label is less valuable than evidence that changes a deployment decision.

Anthropic announced on October 15 that it has onboarded six employees from METR, an outside AI evaluator. The company says their access to internal tools and systems is roughly on par with that of its own risk-assessment teams. [2] [3]

The potential gain is a narrower gap between what outside reviewers can investigate and what the developer itself knows. That could help buyers ask sharper questions about safeguards and incidents. It could also create more work for Anthropic: supporting evaluators and addressing the weaknesses they find are inputs, not free benefits.

Under the arrangement Anthropic describes, it cannot veto METR’s publication. It can require redactions in narrowly predefined categories. Information exclusions include customer data, legally privileged material and model architectures; the access is not unrestricted. [2]

METR can publicly describe Anthropic’s redactions, but cannot reveal the redacted information itself. That gives readers a way to learn about limits on what they can read, without opening the excluded material. [2]

For a hypothetical enterprise buyer, a substantive report could focus technical due diligence on a particular safeguard rather than a broad questionnaire. But if important context is withheld, the buyer may still need its own tests. Wider access alone is no basis for budgeting fewer engineering hours or assuming safer models.

Anthropic says it is embedding evaluators voluntarily. This is the company’s description of the arrangement, not a completed assessment of how well it works. [2]

METR’s own site records earlier internal testing: a March 26 account describes a staff member spending three weeks red-teaming a subset of Anthropic’s internal monitoring and security systems—trying to find ways those systems could fail. [3]

That is the strongest counterpoint to treating the announcement as a breakthrough: outside scrutiny was already happening. A larger embedded presence might add credibility without materially improving oversight. Voluntary access should not be mistaken for government supervision or a safety certificate.

Watch the embedded team’s published assessments: do they identify specific incidents or safeguard weaknesses, explain meaningful limits on access, and lead to a visible response from Anthropic? Compare their scope with the earlier monitoring exercise. Repetition without new insight would weaken the case that deeper access makes a difference.

For buyers, the practical test is whether findings change a deployment choice or help resolve a concrete assurance question. Track the engineering work needed to investigate and close those questions, rather than counting evaluators as a saving. The promise is better-informed decisions; lower costs and effective oversight still have to be demonstrated.