$ cat wiki/papers/2026/2609.09113-saescientist-bench.md
SAEScientist-Bench: Can AI Agents Conduct Autonomous SAE Interpretability Research?
TL;DR
A benchmark that asks whether an agent can do the auditing work, not just the building work. The authors argue that recursive self-improvement research has automated training pipelines while leaving out post-hoc monitoring and auditing — understanding what a model learned — and that mechanistic interpretability is the missing pillar. Given a target concept, an agent must design contrastive probes and navigate a Gemma Scope dictionary of 131K+ features in Gemma-2-9B-IT to find the best feature, scored against curated expert reference features anchored on Neuronpedia across activation rank, concept selectivity on contrastive texts, and causal steering. Across 10 agent configurations and 20 tasks, frontier agents show genuine discovery ability but remain well behind the expert baseline — near expert level at separating a target concept from contrastive controls, lagging substantially in causal generation steering (source).
Authors & Org
Not published in anything read. Code is stated to be at
github.com/Trae1ounG/SAEScientist
(source).
Method
| Element | Value |
|---|---|
| Target model under study | Gemma-2-9B-IT |
| Feature dictionary | Gemma Scope, 131K+ features |
| Agent's task | given a target concept, design contrastive probes and find the optimal feature |
| Reference | curated expert reference features anchored on Neuronpedia |
| Scoring dimensions | activation rank, concept selectivity on contrastive texts, causal steering |
| Scale | 10 agent configurations × 20 tasks |
| **The three scoring dimensions are not interchangeable, and the gap between them is | |
| the result.** Activation rank and selectivity ask whether the agent found the right | |
| feature. Causal steering asks whether using that feature produces the intended | |
| change in generation. The agents are near-expert on the first kind and substantially | |
| behind on the second. |
Results
| Finding | Detail |
|---|---|
| Overall | frontier agents demonstrate genuine discovery capabilities and lead different evaluation dimensions, but remain well behind the expert baseline |
| Where they are close | separating target concepts from contrastive controls |
| Where they are not | causal generation steering — "lagging substantially" |
| Named failure | agents can design contrasts that rule out spurious candidates, but frequently misinterpret experimental measurements |
| **No absolute score, no agent named, and no per-configuration table appears in | |
| anything read** (source). |
Significance
It gives Mechanistic Interpretability its first agent-capability measurement. Every entry on that page to date measures what interpretability tools can find in a model. This measures whether a model can operate those tools — which is the prerequisite for any of the automated-auditing proposals this wiki tracks, and which nothing here had a number for.
The failure it isolates is the one that matters for auditing. "Agents frequently misinterpret experimental measurements" is not a capability shortfall in probe design; it is a shortfall in reading results. An auditor that runs the right experiment and draws the wrong conclusion from it is worse than no auditor, because it produces a clean report. This is the same shape as the finding on Steering Geometry: Validating Human Value Geometry in LLM Steering Space, captured on 2026-09-10: a method that works and a method that works for the stated reason are different things, and only one of them is what a downstream user is relying on.
It also sits directly against the wiki's newest RSI entry. NeoHorse-1: Towards Recursive Self-Improvement via Agentic Post-Training with Routing Harness frames recursive self-improvement as a training-data pipeline that closes a loop through evaluation feedback. This paper's opening claim is that such loops are missing their audit half — and then measures how far the audit half is from working. Read together they are an argument and its strongest objection, published three days apart, neither citing the other; the pairing is this wiki's.
And it is the third paper in one snapshot about not trusting a result. SWE-Bench Pro Verified: A Reliable Benchmark for Software Engineering Agents says the score was inflated; Scores Alone Do Not Prove Discovery: The Discovery Certification Protocol for Auditing AI Research Agents says a correct score is not a discovery; this says the agent that checks your work may misread its own instruments. Nothing read connects them, and this grouping is this wiki's.
Open Questions
- How far behind is "well behind"? No baseline gap is quantified in anything read, and the whole claim is comparative.
- Which frontier agents were tested, and whether any of the 10 configurations is a model this wiki holds a page for.
- Whether causal steering is hard for agents or hard in general. The expert baseline is a human using the same dictionary; a large human–agent gap on steering could be an agent limitation or an artefact of how steering is scored.
- Whether "misinterpret experimental measurements" is correctable by scaffolding — a verifier over the agent's own conclusions — or is a property of the model.
- Whether the finding transfers off Gemma Scope. One dictionary, one 9B model.
Cite
arXiv:2609.09113 — SAEScientist-Bench: Can AI Agents Conduct Autonomous SAE Interpretability Research? Published 2026-09-08; surfaced in HuggingFace Daily Papers on 2026-09-11 with 16 upvotes — a popularity signal from that community and nothing more (source).