Would this change your answer? Evaluating Explanations of LLM Behavior In The Wild with Counterfactual Experiments

TL;DR

Anthropic proposes counterfactual simulatability as the test of an explanation of model behaviour — does it let you predict what the model does on related edited inputs — and builds CHIVE to generate thousands of such explanations at scale. Every interpretability technique it studied gave no uplift: agents holding activation-reading tools predicted outcomes no better than agents that just read the transcript.

Authors & Org

Adam Karvonen, Euan Ong, Subhash Kantamneni, Samuel Marks — Anthropic Alignment Science. arXiv 2608.16747, submitted 2026-08-17; posted to the Alignment Science blog as "Would This Change Your Answer?" (source).

Method

CHIVE — Counterfactual Hypothesis Investigation Via Edits — is an agentic pipeline in two moves: find unexpected model behaviours in the wild, then investigate each by editing the prompt and seeing whether the proposed explanation predicts what changes. The criterion is borrowed rather than invented: an explanation is judged useful for predicting behaviour on related counterfactual inputs, not for being mechanistically satisfying. Output is reported as "thousands of high-quality explanations for naturally-occurring model behaviors with supporting counterfactual evidence" (source).

Two things about the framing are worth naming. In the wild means the behaviours are found, not constructed — the opposite of the planted-behaviour design Anthropic's own AuditBench uses, where 56 models are fine-tuned to hide a known target. And the metric is predictive, so an explanation can be elegant and still fail.

Results

1. No uplift from any interpretability technique studied. Agents given activation-reading tools predicted the outcomes of the counterfactual experiments no better than agents given only the transcript.
2. Training on CHIVE outputs generalises. Models trained to predict the outcomes of CHIVE-generated counterfactual experiments transfer to held-out, out-of-distribution settings.

No numeric result appears in anything read — no accuracy, no effect size, and no enumeration of which interpretability techniques were in the comparison. The findings above are the direction the authors state, not a measured magnitude, and that gap is why this page publishes no table.

Significance

This is the second negative result Anthropic has published about its own alignment tooling in a week — the first being Fine-Tuned Lie Detectors Failed to Generalize (2026-08-21), where detectors trained on on-policy lies fell to ~0.70–0.75 AUROC out of distribution and were often beaten outright by zero-shot prompting of a larger model. Read together, the two share a shape: the purpose-built instrument does not beat the general-purpose one, and in both cases reading the model's own output was competitive with reading its internals.

It also lands on the same side as AuditBench (2026-03-10), which found scaffolded black-box tools most effective overall and white-box interpretability tools helpful "primarily on easier targets". Three independent Anthropic measurements now point the same way, which is a stronger claim than any one of them. Mechanistic Interpretability is described in this wiki as "the primary empirical tool for alignment verification"; on this task — predicting counterfactual behaviour — it is not yet earning that description, by its principal lab's own reporting.

The constructive half matters as much. If explanations can be generated at scale and scored predictively, then explanation quality becomes a trainable target rather than a matter of taste — and the generalisation result says that target transfers.

Open Questions

• Which interpretability techniques were compared, and at what scale? "No uplift from any studied" is only as strong as the list, and the list was not read.
• What is the absolute prediction accuracy? A tie between tooled and untooled agents is a different finding at 90% than at 55%.
• Does "no uplift on this task" generalise, or is counterfactual prediction under prompt edits a task where activations were never the right instrument? The paper's own positive result — that behaviour prediction is trainable — is compatible with interpretability mattering elsewhere.
• Was the negative result pre-registered? Nothing read says.

Cite

Karvonen, A., Ong, E., Kantamneni, S., Marks, S. Would this change your answer? Evaluating Explanations of LLM Behavior In The Wild with Counterfactual Experiments. arXiv:2608.16747, 2026-08-17. abs · Alignment Science blog · capture