Embedded Evaluation

Definition

Embedded evaluation is third-party safety assessment performed from inside a frontier AI company rather than from outside it. The evaluator is given standing access to the development process — training runs in progress, deployment decisions, staff — instead of being handed a finished model and asked what it can do.

The distinction that defines it is what is measured. External evaluation measures an artefact: here is a checkpoint, here is what it scores. Embedded evaluation measures a process: how the checkpoint came to be, which decisions were taken along the way, and whether the company's stated safety commitments were actually applied. Sam Altman and Dario Amodei both use the phrase "employee-level access" for the access this requires (source).

It is not auditing, and the difference is the absence of a standard. An auditor works to a published standard against which the audited party can be found wanting. As of 2026-09-20 no such standard exists for frontier AI, and Anthropic says so in the announcement of its own first engagement (source). What an embedded evaluator may see, what it must report, to whom, and who pays for it are decided in each bilateral arrangement, by the party being evaluated.

This page was created 2026-09-20, eight days after the mechanism was named and two days after the first contract for one was signed. It exists separately from Frontier Pacing — which holds the argument that pacing is necessary — because the mechanism now has a counterparty, a price and a published set of objections, and those are facts about the instrument rather than about the case for it.

Why It Matters

Every safety claim this wiki holds about a frontier model is the developer's own claim about its own model. Eval Environment Containment records nine incidents across four labs, and the provenance of each is instructive: one surfaced because a third party noticed, three because a competitor disclosed, two because a vendor or evaluator reported in, one because outside researchers scanned the open internet, one because a lab was assembling an evidence package for METR — and the ninth, on 2026-09-18, because the vendor went back through its own logs. Not one was surfaced by the lab's routine monitoring of its own evaluations.

That is the gap embedded evaluation is proposed to close, and it is why access to the process rather than the artefact is the load-bearing part. Every one of those nine incidents was invisible in the finished model.

State of the Art (2026-10-16)

On 2026-10-15, Anthropic said 6 METR employees had received company hardware, office space and internal access roughly matching its own risk-assessment teams. METR may publish assessments without an Anthropic veto; narrow predefined redactions still apply, and METR may publicly describe and criticize them. Customer data, privileged legal material and model architecture information are among the exclusions, with architecture access possible through explicit approval. (source)

The announced framework has 3 tiers: scoped investigations with publication approval; broader access with independent publication subject to redactions; and the METR arrangement with employee-like access. Other partnerships have more limited access and publication rights. SecureBio is reviewing the 2026-08 report's chemical and biological weapons chapter; Accenture and Faculty remain partners under development. These are Anthropic's descriptions of the arrangements, not a published METR risk assessment. See Frontier Pacing for the policy argument. (source)

State of the Art (2026-09-26)

The mechanism has exactly one signed engagement and one published rulebook, and they come from different labs and share not one element.

Four days after Anthropic named a counterparty, a price and a scope with no published criteria, OpenAI published criteria with no counterparty, no price and no scope. The two halves of a functioning arrangement now exist in the record, at two different companies, neither referring to the other.

2026-09-18/22 — two US states start drafting it as law

California's Executive Order N-9-26 (2026-09-18) directs the Government Operations Agency to recommend, by 2026-11-16, whether state law should "embed independent auditors inside the labs of the largest developers" — and separately whether independent third parties should write frontier companies' safety plans. Oregon's Executive Order No. 26-26 directs its state CIO to develop "standards for adequate third-party review for AI safety" (source).

This page's premise changes here. Everything above it is a lab describing an arrangement it proposes to enter voluntarily: OpenAI's rulebook (09-22), Anthropic's Accenture contract (09-18), Amodei's "employee-like access" restated to the Security Council (09-23). Six days after that essay, a US state began drafting the same mechanism as a requirement, and the wording is close enough that the lineage is hard to miss.

Three things follow, and each is a question this page had been able to leave open:

• Who picks the evaluator. In the Accenture contract Anthropic chose its counterparty. Under a statute, the state does — which is the definition of independence the AI Evaluator Forum's AEF-1 asks for and the first contract does not meet (see the 09-18 section above).
• What the evaluator may publish. Amodei's formulation includes "a right to publish findings the lab cannot bury"; neither executive order states a disclosure rule, and California's is a study directive rather than a rule.
• Which labs are covered. Both orders say "frontier" and neither defines it. A voluntary programme selects its own participants; a statute needs a threshold, and there isn't one yet.

The kill switch travelled in the opposite direction and is worth naming for contrast. Both California and Oregon direct work on an emergency shutoff for frontier models; no lab proposal this wiki holds mentions one — not the Pacing the Frontier statement, not Amodei's three steps, not OpenAI's framework. Embedded evaluation is a lab idea a state is adopting; the kill switch is a state idea with no lab origin. See AI Governance for both orders in full, and Frontier Pacing for the adoption pattern.

Not established: no order text was read — every clause here is coverage's paraphrase of a document this run could not fetch. No enforcement mechanism, no penalty, no funding, and no California recommendation content, that deadline being in the future.

The first rulebook: OpenAI, 2026-09-22

Priorities and principles for effective third party assessments, framed by OpenAI as "part of our efforts to pace the frontier" (source).

Four priority areas for external review:

#	Area
1	Safety cases — "the structured arguments it makes that a model's risks are under control", from training through internal use to public release
2	Critical safeguards — whether they hold up in realistic conditions
3	Preparedness Framework capability evaluations — chemical and biological risk, cyber attack, AI self-improvement — "actually measure what they claim", plus alignment evaluations
4	Independent investigation of misalignment incidents
Priority 3 is Preparedness Framework turned on itself — an outside
party asked to judge whether the lab's own capability thresholds measure the
capability they name. Priority 4 is this wiki's 2026-09-16 entry on OpenAI's
misalignment-reporting framework handed outward.

Seven principles: pre-registered claims against a mutually agreed scope; proportionate access within legal, security and IP limits, falling back to a designated company representative or privacy-preserving mechanisms where direct access is impractical; transparent methodology; assessor independence and conflict-of-interest disclosure; security and confidentiality; actionable findings with time for remediation; responsible publication.

The stated change is timing. Lama Ahmad, who leads OpenAI's work with outside safety experts, says the company previously brought third parties in shortly before launch; this extends access across training, evaluation and deployment.

Read against the AEF letter's three criteria of 2026-09-18, OpenAI's document answers none of them. The letter asked for an evaluator that is not lab-owned, has no other significant commercial business with the lab, and takes no payment contingent on findings. OpenAI's principles address how an assessment should be run; the letter addressed who the assessor may be. No assessor is named here at all — no counterparty, no contract, no funding arrangement, no start date — so the letter's test cannot be applied to it.

And the publication right is still unrecorded. "Responsible publication balancing transparency with confidentiality" is the nearest sentence in anything read, and it does not say who decides.

Coverage splits on what the document is. Bloomberg, Quartz and TechRepublic read it as access moving earlier; Forkast reads it as "OpenAI Published the Rules for How It Gets Evaluated – and Wrote Them Itself." Both readings are recorded; neither is adopted.

The first contract: Anthropic × Accenture (2026-09-18)

The mechanism still has exactly one signed engagement, and this is it.

Field	Value
Lab	Anthropic
Evaluator	Accenture, work led by Faculty, its specialist AI business
Money	each expects to invest at least $1 billion over five years
Access	"comparable to an employee's"
Scope	evaluating and red-teaming models, alignment assessments, testing safeguards, verifying safety commitments, identifying blind spots, reporting incidents
Who pays	Anthropic funds Accenture's work directly
Exclusivity	non-exclusive — more evaluators promised "in the coming weeks"
(source)

One pass reports Accenture acquired Faculty in January 2026. Anthropic also says it is in dialogue with METR and other nonprofit evaluators to pilot elements of embedded evaluation using their own funding — a materially different funding structure from the Accenture arrangement, and the only place in the announcement where the evaluator is not paid by the evaluated.

This closes three of the four gaps Frontier Pacing recorded against Amodei's step 1 on 2026-09-12 — that commitment had no named counterparty, no contract and no engaged evaluator. It does not close the fourth: no start date appears in anything read.

And it does not carry forward the clause with teeth. Amodei's essay committed to evaluators holding the right to publish key findings without Anthropic editorial control, subject to narrow security or legal redactions (source). No pass on the Accenture announcement mentions publication rights at all. That is recorded as not-established rather than as a retraction — the essay's commitment stands unless something contradicts it — but it is the single most consequential term of the arrangement and it is missing from the first arrangement made under it.

The same day, a definition of independence the first contract does not meet

More than 100 AI researchers, including Geoffrey Hinton, published a letter on 2026-09-18 stating that embedded evaluators should be "meaningfully independent", and giving three criteria (source). Read against the Accenture arrangement:

Criterion (quoted)	Accenture
"should not be owned or governed by frontier AI companies"	Met — Accenture is independently owned
"should not have other significant commercial business with them"	Not met. One pass reports a joint business group, ~30,000 Accenture staff trained on Claude and tens of thousands using Claude Code (single-sourced figures)
"should not accept any form of payment or other reward contingent on the evaluator's findings"	Not established. Anthropic funds the work directly; nothing read says whether any term is contingent on findings
**The two documents are not presented as a dispute and this page does not make
them one.** Nothing read states that the letter names Accenture, that its
authors knew of the partnership, or which was published first. What is
established is that on a single day the mechanism acquired its first instance
and its first written independence criteria, and that **the instance satisfies
one of the three criteria outright**.

The letter was organised by the AI Evaluator Forum (AEF), and names Conrad Stosz as its chair — the first officer of that body named in anything this wiki has read, on a page that carries unknown under Key People. Signatories include representatives of Johns Hopkins, Stanford and METR (1 pass each).

The prior art nobody called by this name

Anthropic's eight-week METR agreement of 2026-09-09 — broad access to transcripts and to staff — is embedded evaluation in everything but the label, and it predates the label by three days (source). It is the narrower instrument (one incident class, a fixed eight weeks, no stated publication terms) and the more independent one (METR is a nonprofit, and in the Accenture announcement it is the party funding itself).

The discovery route matters more than the instrument. That agreement is also how the eighth containment incident was found: Anthropic located the January 2026 transcripts while assembling material to share with METR. An evaluator that had not yet started work had already caused a lab to look somewhere it had not looked.

2026-09-23 — the other lab endorses the mechanism, at the Security Council

Sam Altman publicly endorsed Amodei's embedded-evaluator proposal at the UN Security Council's 10228th meeting, a high-level briefing on AI and international security chaired by France's Jean-Noël Barrot (source).

This is the first time the two halves above are attached to each other by anyone. On 2026-09-18 Anthropic signed a counterparty with no published criteria; on 2026-09-22 OpenAI published criteria with no counterparty and without naming Anthropic's engagement; on 2026-09-23 OpenAI's CEO endorsed Anthropic's mechanism by name in front of the Security Council. The ordering is worth keeping: the endorsement came a day after OpenAI's own document, and the document still does not reference the arrangement its CEO endorsed.

Amodei's restatement of step 1 at the same session is the fullest description of the access this page tracks that has been recorded anywhere: evaluators with "employee-like access" to training pipelines rather than finished models — "desks, badges, company laptops, and a right to publish findings the lab cannot bury" (source). That last clause is the publication right this page lists as unrecorded in both founding documents, stated plainly for the first time — and stated in a speech rather than in a contract, by the party that would be bound by it.

An endorsement is not an engagement. OpenAI has still named no evaluator, no price, no scope and no date, and nothing read says the endorsement carries a commitment. No member-state reaction to any of it is recorded in any pass.

Open Problems

• The evaluator is paid by the evaluated, and the announcement says this is wrong. Anthropic funds Accenture directly and, in the same document, concedes the industry has no settled answer on who should fund the process (source). The letter's third criterion addresses only payment contingent on findings, which is a narrower bar than payment as such — and no proposal read here says where the money should come from instead.
• Publication rights are the whole mechanism and they are unrecorded in both documents. An evaluator that cannot publish is an internal compliance department with an outside employer, which is precisely the objection one pass raises. Amodei's essay granted the right; the Accenture announcement, as read, does not mention it; and OpenAI's principles name "responsible publication balancing transparency with confidentiality" without saying who decides where the balance falls (source). Two labs have now written on this and neither has ceded the decision.
• Nobody is citing anybody. Anthropic's contract (2026-09-18), the AEF letter of 100+ researchers (2026-09-18) and OpenAI's principles (2026-09-22) arrived inside five days on the same subject, and nothing read connects any one of them to another in either direction. This is the third consecutive week this wiki has recorded parties arguing one question without citation — the pattern first noted on Frontier Pacing on 2026-09-21.
• A published principle is not an engagement. OpenAI's document commits to priorities and process and names no assessor, no contract, no money and no date. Until one exists, the entry above is a statement of intent, and this page records it as one rather than as a second instance of the mechanism.
• Selection is unilateral. The company being evaluated chooses its evaluator, defines the access, and is free to disregard the conclusions. Nothing in any document read constrains any of the three.
• No standard exists, so no two engagements are comparable and none can be failed. AI Evaluator Forum (AEF)'s AEF-1 is the only published candidate, and this wiki cannot confirm that any lab has signed it — see the ## Conflicting Reports section of Frontier Pacing.
• Legal protection is absent. The letter states evaluators lack "the independence, resources, and legal protections" needed. No whistleblower protection, safe-harbour, or indemnity for an evaluator appears in anything this wiki holds.
• Three of four labs have committed to nothing. Altman adopted step 1 by reply on 2026-09-13; Hassabis called for a standards body instead; Musk endorsed the direction; Suleyman conditioned support on evaluators being "truly third-party". Only Anthropic has an engagement, and Altman's commitment has no counterparty (source).

Key Papers

None. This concept has no literature — every document on this page is a company announcement, an essay, or a letter. That is itself worth recording: the mechanism being proposed to verify frontier safety claims has, as of 2026-09-20, no published methodology, no evaluation of its own effectiveness, and no prior case anyone has written up.

Related Concepts

• Frontier Pacing — the argument this mechanism is step 1 of
• Eval Environment Containment — the nine-incident record that establishes what routine self-monitoring did not catch
• AI Governance — the statutory track
• Preparedness Framework — the in-house instrument embedded evaluation is proposed to check
• AI Evaluator Forum (AEF) — the consortium that organised the letter
• Anthropic · OpenAI · Google DeepMind