2026-09-01

LoopArena: Benchmarking Models as Runtime Controllers for Loop Engineering — arXiv:2608.28281 — score 1.00

• TL;DR: when a coding agent fails a long task, nothing tells you whether the loop driving it guided badly or the agent executed badly. LoopArena fixes the coding agent (the Worker) and evaluates the model instructing it (the Controller), naming the practice it measures — Loop Engineering, designing a loop that monitors progress, assigns work, runs checks and decides what comes next instead of writing each prompt.
• Best observed Strict Success Rate on full tasks: 24.69%. Mean paired inference-cost reduction: 64.4%. Its cheap setting reproduces the expensive one's ordering at Spearman's ρ = 0.9747 — a rank claim, which does not license quoting a cheap score as a full-task score.
• Why read it: Eval Harness Configuration exists on this wiki because vendors publish agentic figures without naming the scaffold. Every entry there treats the harness as a nuisance parameter to disclose. This inverts it — the harness's controller becomes the capability under test — and it is the first benchmark here to do so.
• The gap that undercuts it: no model is named anywhere in the abstract — not the best Controller, not the fixed Worker. A benchmark built to disentangle two systems, published without identifying either, cannot be tracked across releases.
• → LoopArena: Benchmarking Models as Runtime Controllers for Loop Engineering

ContextPilot: Teaching Agents for Proactive Context Management via Fine-grained RL — arXiv:2608.28476 — score 0.90

• TL;DR: long-horizon agents accumulate an ever-growing working context. Prior work lets a model edit its own context with search, delete and summarize tools; ContextPilot adds planning, long-term memory and soft offloading — and then replaces the trajectory-level reward with action-level advantages for individual context edits, estimated by branching at decisions flagged by context and entropy variation.
• Reported as stronger and more compact on long-context QA and deep search. From Tencent, code released.
• Why read it: the credit-assignment argument generalises well past context management, and it is the third result this month making it — SecOPD: Mitigating Adaptive Prompt Injections by On-Policy Distillation (arXiv:2608.21500) and OPDVR reach it through token-level distillation, ContextPilot through branching. Read beside yesterday's WikiSkill: Compiling Agent Experience into Persistent Knowledge for Skill Evolution, the two answer one problem at different layers: write the scattered information into a durable external artefact, or learn in flight what to throw away. Dated a day apart, neither cites the other.
• Caveat that limits all of it: the abstract names no benchmark, no model, no baseline and no figure. Direction only.
• → ContextPilot: Teaching Agents for Proactive Context Management via Fine-grained RL

J-Zero: Unified Challenger–Solver–Judge Co-Evolution from Zero Data — arXiv:2608.26582 — score 0.84

• TL;DR: self-evolving models have progressed where an automatic verifier can score the answer and much less where one cannot. J-Zero co-evolves a Challenger, a Solver and a Judge, and takes the Judge's preference labels from how each response was produced — the Solver's answer over the Challenger's, a decomposed-and-recombined answer over a one-shot one — rather than from the Judge's own scores. +4.2 points verifiable · +8.0 unverifiable, still improving at ten iterations where baselines degrade after two.
• Why read it: it lands on a line this wiki drew three days ago. AI Alignment holds Anthropic publishing both halves in August — automated researchers beating 28 experienced humans where "an objective benchmark, not a fallible human, decides whether a fix works", and TASTE at 60% against 77% human agreement with Opus 5 and GPT-5.6-Sol near chance. J-Zero reports its larger gain on the side that pair said was hardest, and it is the first datapoint here pushing that way.
• It is not a refutation and the page does not call it one: TASTE measures a frozen model judging, J-Zero trains a judge.
• The unanswered question is the load-bearing one: if the Judge is part of the system being improved, something else had to score the "unverifiable" evaluation — and nothing read says what. No benchmark, base model or baseline is named either.
• → J-Zero: Unified Challenger–Solver–Judge Co-Evolution from Zero Data

• A Tier-1 source published today and this run could not read a word of it. Import AI 471 (Jack Clark, 2026-08-31) arrived as a prefetch candidate, but jack-clark.net and importai.substack.com both answer EGRESS_BLOCKED, and two search passes returned a garbled account of its lead item that conflated Clark's commentary with the OpenAI–Hugging Face incident this wiki already holds in detail. It was not written from. A newsletter whose body cannot be read is not a licence to write from a summary of it — and this is the same source sources.yaml recorded as listed, cited 17 times and never delivering an item until the RSS mirror was added on 2026-08-02
• 2608.27370 Puro-2B is the strongest paper this brief declined, and the reason is that nothing here connects to it yet: a full open pretraining recipe — 1.4 trillion tokens, FP8, consumer RTX 5090 GPUs, best model under $6.9K of compute, Apache 2.0 on data, code and weights — plus a fitted Puro Cost Scaling Law putting Qwen2-1.5B parity near $4.4K. Open-source and local inference weigh 1.0 here. If a second result lands on training-cost floors, this becomes a page
• The Anthropic Alignment Science standing check could not run for a fourth consecutive day. alignment.anthropic.com answers EGRESS_BLOCKED, so the index was not read at all. Introspection Adapters (April 2026) and The Hot Mess of AI (February 2026) remain uncaptured at day 21. Recorded as unreadable-this-run, not as clear
• 2608.24777 StepGuard was the closest miss on the paper picks: a step-level guard auditing tool calls before execution, reporting a 77.3% mean attack-success-rate reduction on AgentDojo and AgentDyn for a 2.8 percentage point utility cost, and the highest average accuracy among open-weight guard models. Held because SecOPD: Mitigating Adaptive Prompt Injections by On-Policy Distillation (arXiv:2608.21500) already carries this month's prompt-injection result; a candidate the moment that cluster grows again

No new entity or concept page today, so nothing needs user review.

• LoopArena: Benchmarking Models as Runtime Controllers for Loop Engineering (new — paper page)
• ContextPilot: Teaching Agents for Proactive Context Management via Fine-grained RL (new — paper page)
• J-Zero: Unified Challenger–Solver–Judge Co-Evolution from Zero Data (new — paper page)

• OpenAI: the first OpenAI-published advertising revenue figure, the tier policy, and the two quantities left unresolved rather than reconciled
• Anthropic: the CCTV-affiliated precondition, marked as commentary rather than a ministry position, with the unadopted fingerprinting allegation named as unadopted
• AI Governance: two new sections — a named lab's conduct as a bilateral bargaining chip, and a consumer-scale advertising product entering the jurisdictions whose AI Act obligations this page already tracks
• Eval Harness Configuration: the harness stops being the nuisance variable and becomes the subject
• Agents (LLM Agents): the loop-controller inversion, and context management as a trained policy
• Agentic Reinforcement Learning: J-Zero and ContextPilot added to Key Papers, the second on the same credit-assignment complaint as SecOPD
• AI Alignment: the first result here claiming movement on the unverifiable side of the AAR/TASTE boundary
• Tencent: the lab's first agent and evaluation research recorded here, after four entries of generation artefacts