AI Trend Notifier
EN
← archive

$ cat briefs/daily/2026-09-16.md

2026-09-16

September 16, 2026 (Wed)

2 stories · 2 paper picks · 3 watch items · 5 new pages

**On Sunday Demis Hassabis answered Anthropic's evaluator proposal by saying what was really needed was an industry-wide standards body, and nobody in the conversation mentioned that one had been running since December.** It has a membership, a launch date and a published standard — AEF-1, minimum operating conditions for independent third-party evaluations — whose load-bearing clause is nearly word-for-word the one in Amodei's essay. Separately, Google DeepMind shipped two voice models, and the one that is generally available published no benchmark figure at all. The papers measure the two ends of the same loop: how much of the work agents are now doing, and where the returns on letting them keep working go negative.

+5new pages
[01]

Top Stories

1. The standards body the pacing debate asked for on Sunday had already existed for nine months, and had already published a standard (1.86 — published first, out of score order; see the note under Story 2)

  • AI Evaluator Forum (AEF) — a consortium of independent evaluators, formed December 2025, launched 2025-12-04 co-located with NeurIPS25. Founding members include Transluce, METR, RAND and SecureBio (source)
  • Its first output is AEF-1 — "Minimum Operating Conditions for Independent Third Party AI Evaluations": a voluntary standard whose purpose is that third-party evaluations be run under conditions ensuring independence, access and transparency, and which evaluators "can use to demonstrate how they achieved a baseline set of operating conditions"
  • What it is reported to require — two one-pass summaries, recorded as summaries because no pass returned clause text: sufficient technical access to assess the characteristics under evaluation; access to system prompts, training process information, pre-existing internal evaluation results and knowledge of system vulnerabilities; and provisions for editorial control over methods and results, removing conflicts of interest and safeguarding intellectual property
  • Why it matters: Frontier Pacing has held a form with no body since 2026-09-06 and a body with no form since 2026-09-13. This is the first object that is both — and it inverts who is being bound. Amodei's step 1 is a grant from a lab, revocable by the lab that granted it; AEF-1 is a disclosure by an evaluator, which no lab can withdraw and no lab has to accept. Neither binds a company to anything
  • The overlap on the clause with teeth is close enough to state. The essay's is the right to publish key findings without Anthropic editorial control; AEF-1's is editorial control over methods and results. Four days and one consortium apart, and nothing read reports either citing the other
  • What is not established, and it includes the headline. The item that surfaced this asserts that xAI, OpenAI and Anthropic all cosigned AEF-1. Four passes surfaced that same headline; none corroborated it independently, and none produced a signatory list, a date of signature, or any statement by the three labs about AEF-1. A pass asked directly returned only their replies to Amodei's essay — a different object, four days earlier, with three of the same names. Also unestablished: whether this is the body Hassabis meant (Google DeepMind is in neither reported founding-member list), any effective date or version date, and any enforcement mechanism, audit or registry
  • No first-party readaievaluatorforum.org and www.aef.one both answer EGRESS_BLOCKED and are new to this repo's list
  • Frontier Pacing · AI Evaluator Forum (AEF) · Google DeepMind

2. Google DeepMind shipped two voice models and only the paid-reasoning one has a number (1.93)

  • 2026-09-15, Introducing Gemini 3.8 Live and 3.8 Live Extended Thinking — rolling out "starting today" through the Gemini API, Google AI Studio and Search Live, enterprise via Gemini Enterprise (source)
  • Stated for the pair: switching between 97 languages mid-conversation, tool calls and API requests executed in the background while the model keeps talking, and visual input processed in near real time
  • Gemini 3.8 Live Extended Thinking tops the Artificial Analysis Speech to Speech Quality Index at 82.6%, against GPT-Live-1 "Astra (Medium)" at 81.5% and Grok Voice Think Fast 2.0 "(High)" at 81.3%, and costs $3.50 per hour of input audio against $5.83 and $4.80
  • Gemini 3.8 Live — the generally available, cost-efficiency half — has no published figure of any kind
  • Why it matters: two of the three benchmark rows carry a reasoning-effort qualifier and the winning row carries none, so a 1.1-point lead is published with no statement that the three were run at matched effort. That is the exact distinction Eval Harness Configuration exists for, and it is the second Gemini release in a fortnight where the number that travels is the one hardest to check
  • The price is per hour of audio, and that has a consequence here. scripts/spec-check.py reads Pricing back against a per-token catalogue, so an audio-hour rate is a cell it cannot verify — the same blind spot Gemini Omni 1.1 Flash carries with per-second video pricing
  • What is not established: no per-token price, context window, maximum output or API model id for either model. A pass asked directly for per-million-token pricing returned Gemini 3.8 Flash's figures and said so; those belong to a model released thirteen days earlier. No first-party readdeepmind.google and blog.google both answer EGRESS_BLOCKED, and the announcement's URL, title and date come from DeepMind's RSS via state/prefetch.json
  • It outscores Story 1 and is published second. 1.2 × 1.3 × Google DeepMind 1.3, −0.1 for an already-rich area, against 1.2 × 1.3 × 1.0 + 0.3 for a new entity page. AEF-1 leads because it answers a standing Open Problem on a page this wiki has built since July, while a voice-model release is the more routine event; both scores are shown and the inversion is stated, as on 09-04, 09-06, 09-14 and 09-15
  • Gemini 3.8 Live · Gemini 3.8 Live Extended Thinking · Google DeepMind
[02]

Paper Picks

Both picks score 1.85 and the tie is real — base 1.3 × agents 1.5, −0.1 each, with nothing in interests.md able to separate them. This is the fourth distinct scoring collision recorded in fifteen days; as on 09-09, 09-11, 09-12 and 09-15, no weight was adjusted mid-run.

When Agents Slow Down: Understanding LLM Agents' Test-Time Strategies via Elo-per-token AnalysisarXiv 2609.15309 (1.85)

  • TL;DR: track the best solution found at each token budget, aggregate within-task orderings into Elo via Bradley-Terry, and compare against independent sampling, whose Elo grows linearly with log compute. Four general-purpose agents on four open-ended benchmarks, sessions to 100M tokens: they convert tokens into Elo faster than the reference at first, diminish, and eventually fall below it. The crossover is named the scaling inflection point
  • Why read it: it is a prescription, not just a diagnosis. Splitting 100M tokens into parallel sessions sized at that point gained +264 Elo over one long session and +355 over ten short ones on FrontierCS Polyomino Packing
  • The decaying thing is the scaffold, which is why it lands on Agents (LLM Agents) as much as on Test-Time Compute (Inference-Time Compute Scaling): an agent revising, exploring and deciding when to stop is what eventually underperforms sampling and keeping the best. The strongest human contestants, on shared AtCoder Heuristic Contest tasks, improve superlinearly — differently shaped curves on the same tasks, and the human one keeps bending upward
  • Neither the four agents nor the four benchmarks is named, no inflection value is published for any of them, and the +264 / +355 pair comes from one task
  • When Agents Slow Down: Understanding LLM Agents' Test-Time Strategies via Elo-per-token Analysis (new)

Atria Dawn: The Dawn of Agentic SuperintelligencearXiv 2609.15818 (1.85)

  • TL;DR: an agentic foundation model trained through a Verifiable Experience Pipeline — tool-mediated interactions against executable environments and externally verified outcomes — reporting 16 benchmarks, the highest reported score on five, and "competitive" elsewhere. Published alongside a study of 769 task records from 56 participants in its own development: about one-third of completed AI-assisted tasks were rated infeasible without AI, and agents "frequently propose methods and implement revisions" while humans "retain most final decisions"
  • Why read it: Frontier Pacing's only figure of this shape is OpenAI's 3.1 agent-workdays per human workday, which is an input measure with no output attached and no published methodology. This is the output side, with a stated denominator — and it is a self-study: the raters built the system they rate, and no control, blind condition or external replication appears in anything read
  • A paper titled for superintelligence closes by asking for human oversight, arguing progress must advance "both the capacity for discovery and the capacity for meaningful human oversight, preserving accountable human authority". "Most final decisions" is not a number, which is the same gap AI Control Roadmap keeps arriving at
  • No lab, author, weights, licence, parameter count or benchmark list is published — for a paper whose central evidence is an internal labour study of its own team, the missing affiliation is material rather than clerical
  • Atria Dawn: The Dawn of Agentic Superintelligence (new)
[03]

Watch

  • Step 3 may have a meeting date, and this wiki has no record of it. Chasing a Trivium China item on China's AI-safety positions surfaced a Reuters report of 2026-09-04 that the US and China are preparing a mid-September AI safety dialogue, with Treasury Secretary Scott Bessent leading the US delegation, following the May 2026 Trump–Xi summit. Frontier Pacing records step 3 — bargaining with authoritarian governments — as having no taker, and a scheduled bilateral is the nearest thing to a counterparty this argument has had. Nothing read establishes that it happened, what was discussed, or that pacing was on the agenda, so no page was changed. Bessent appears on AI Governance only in the July distillation-sanctions context
  • Three recursive-self-improvement papers landed in one 25-entry snapshot, in the week RSI is the stated first reason for slowing down. Besides today's second pick: 2609.14858 Dream-RSI, self-improving exploration via a replay simulator built from historical discovery trees, and 2609.15364 RSIAgent, whose frozen memory is stated to let Kimi K3 and GLM-5.3 outperform GPT-6 on OSWorld-v2 and Agent's Last Exam. Neither was given a page; both are cited to the snapshot rather than wikilinked, so nothing links to a page that does not exist. Nothing read connects any of them to the pacing argument — the cluster is the observation
  • A cheapest-inference claim was reported to be a resale, and this repo has a script that assumes exactly that problem. A r/LocalLLaMA thread (prefetch #41, 2026-09-15) reports CrofAI, advertised as the world's cheapest inference provider, exposed as an OpenRouter wrapper. CLAUDE.md already documents why spec-check compares against the first-party provider's endpoint rather than a catalogue headline — GLM-5.2 alone is served by 33 providers between $0.72 and $2.31 on input. Single-thread sourcing, so no page was changed; flagged against Model Routing
[04]

New in Wiki

Created today. Flagged for your review.

The +0.3 applied once. AI Evaluator Forum (AEF) is a genuinely new entity; the two model pages are not entity or concept pages, so the "new entity/concept page needed" adjustment does not reach them, which is the literal reading interests.md supports and the same reading the 09-11 and 09-15 runs made.

[05]

Updates

  • Frontier Pacing — new section on the Forum and AEF-1, Open Problem 1 advanced (a forum and a body now exist, and neither answers steps 2 or 3), and a new ## Conflicting Reports entry on whether three labs cosigned the standard
  • Google DeepMind — the two Live models added to Models & Products, plus a Recent Activity entry recording that the standards body its own reply pointed to had existed since December and that DeepMind is in neither founding-member list
  • Agents (LLM Agents) — both papers added to Key Papers / Events
  • Test-Time Compute (Inference-Time Compute Scaling) — Elo-per-token added to Key Papers; the page's first published stopping point rather than another scaling result
  • index.md — five new lines