Evaluators from Faculty, Accenture's specialist AI business, will work with access comparable to an employee's: they test the models, check the safeguards and can report incidents. Anthropic pays.
- An embedded evaluator sees how models take shape in training and the decisions around releasing them.
- The two companies each expect to invest at least $1 billion over five years.
- There are no standards yet for what evaluators should see or how they should report, in Anthropic's own words.
An auditor paid by the audited. Accounting has known this problem for a long time.
Anthropic itself writes in the announcement that there is no settled system for funding independent evaluation. And pays anyway, with an explanation.
Why it makes sense anyway
An outside evaluator has so far worked from outside, without employee-level access. An embedded one sees the path: how the model takes shape in training, the decisions around release, the conversations with the people inside.
The weak spot is visible in the announcement itself: the money comes from the one being checked. The talks with METR about pilots with their own money are the right direction, but for now they are talks, with no signature and no date.
The whole idea will prove itself with the first report an embedded evaluator publishes that Anthropic does not like. If one comes out, the system works.