2026-09-05

Three papers tie at 1.75 and are published as a set, because the finding is that they partition one object. Eval Harness Configuration argues a benchmark number is a claim about a (model, harness) pair; this snapshot has one paper scaling the environment, one optimising weights and harness jointly, and one showing the prompt is part of the pair too. None of the three cites the others.

WHALE: A Simple Recipe for Joint Harness-Weight Optimization — arXiv:2609.00196

• TL;DR: alternate weight updates under the current harness with harness search under the updated weights; +4.15 to +24.38 pp best mean@8 over weight-only, harness-only and Fast-Slow Training, on Qwen3.5-2B/4B across search QA, math and chess puzzles (source)
• The result is the spread, not the headline: harness search matches peak weight-only accuracy on SearchQA with far fewer rollouts, but improves math accuracy only after a weight update. Either component can be the bottleneck, and which one changes by domain
• Why read it: it runs the controlled comparison this week's three harness papers imply and none performed, and it answers Agentic Reinforcement Learning's standing question — whether agentic RL buys anything a rewritten harness cannot — with "sometimes, and it depends", which is weaker than either camp has been claiming
• → WHALE: A Simple Recipe for Joint Harness-Weight Optimization

Terminal-Universe: Turning Agent Trajectories into Scalable Terminal Environments — arXiv:2609.04148

• TL;DR: replay the file operations recorded in an agent trajectory to restore each file to its pre-modification state, have a completion agent supply what is missing, and recover a reusable executable environment — then scale by breadth (cross-workspace queries spanning multiple codebases) and depth (multi-round sessions with a user agent) (source)
• The framing is the contribution: a trajectory is a single frozen demonstration, while an environment can be re-queried into many verifiable tasks and returns execution feedback — and trajectories are the byproduct every lab running coding agents already holds in quantity
• Why read it: Post-Training Scaling names generated environments as an ingredient of the post-pre-training scaling claim and has never held a published means of producing them. This is one. Read with the caveat that the abstract carries no figures at all — no benchmark, no baseline, no environment count
• → Terminal-Universe: Turning Agent Trajectories into Scalable Terminal Environments

RealSWE: A Compositional Evaluation of Coding Agents under Realistic User Requests — arXiv:2608.27831

• TL;DR: problem-statement-only requests are 88% of real user prompts and 7% of benchmark problems; 87% of real prompts are casual against 94% of benchmark problems being formal. On 381 task families where variants share the task and the gold patch and differ only in prompt, realistic inputs drop resolution 6.4 pp on average and can change model rankings (source)
• The intuitive reading is the one the paper rejects: linguistic style has only small, model-dependent effects. What matters is which information the prompt contains — Desired Behavior and Motivation significantly affect performance, while Environment Information and Reproduction Steps "merely add tokens without measurable benefit"
• Why read it: it adds a configuration variable nobody reports to every SWE-bench figure on this wiki, and the two categories that matter are the two real prompts most often omit — which is actionable for anyone using a coding agent, independent of the benchmark argument
• → RealSWE: A Compositional Evaluation of Coding Agents under Realistic User Requests

• On-policy distillation loses its third assumed-essential ingredient in eight days, and this one is the data — Rethinking On-Policy Distillation of Large Language Models II: One Training Example trains on a single query and recovers most of full-data OPD's gain; state coverage is 71.5% from one query and 98.9% from sixteen, while alignment slows at the same rate either way. The authors' summary is "data-overfed but algorithm-starved". It scored 1.49 and sits here rather than in Paper Picks, below a three-way tie at 1.75, because interests.md weights RL at 1.3 and agents at 1.5 — recorded because by this wiki's reading it is the day's most surprising single result (source)
• Grok 4.7 named at roughly ten days out, with a parameter figure that bears on an open conflict — one pass reports Elon Musk saying on X on 2026-09-02 that xAI's next flagship is about ten days from release at roughly 2.1 trillion parameters, "40% larger" than Grok 4.6's 1.5T. Grok 4.6 carries that parameter count as disputed — 2T announced against 1.5T in launch coverage — and this framing takes the 1.5T side. One secondary pass, no first-party read, and no page changed on it
• China's regulator names five categories of AI security risk and three of them were readable — Wang Lihong of the CAC's Cybersecurity Coordination Bureau, to CCTV on 2026-09-01: inherent technological vulnerabilities (naming lack of interpretability first), misuse and abuse, and technological hegemony. Two of the five were not read and are not guessed at. This partly closes the item the 2026-09-04 brief carried in Watch as unread (source)

No new entity, concept or person page today, so nothing here needs owner review — the section says so rather than being omitted.

• WeatherNext 3 (new — model page)
• Rethinking On-Policy Distillation of Large Language Models II: One Training Example (new — paper page)
• Terminal-Universe: Turning Agent Trajectories into Scalable Terminal Environments (new — paper page)
• WHALE: A Simple Recipe for Joint Harness-Weight Optimization (new — paper page)
• RealSWE: A Compositional Evaluation of Coding Agents under Realistic User Requests (new — paper page)

• Safety Monitoring and Data Retention — the comparison table's Anthropic column rewritten for EFS; the 08-20 reported plan kept rather than deleted, because plan and announcement differ on whether the 30-day requirement survives; Open Problems 3 and 4 updated, 3 answered and 4 sharpened
• Eval Environment Containment — a seventh incident, the first that no lab disclosed and the first whose violated property is that instances could coordinate rather than that one escaped
• Agentic Reinforcement Learning — the tenth OPD result, plus 2609.04094 DRACO recorded without a page: dynamic rubrics redistributed over the steps responsible, +15.9 on AppWorld over base and +5.3 over GRPO trained with a sparse ground-truth reward, using no verifier itself
• Eval Harness Configuration — a new dated section: the prompt distribution is a configuration variable, it is unreported, and it does not preserve model order
• Post-Training Scaling — the environments ingredient gets a published means of production, and WHALE constrains the scaling claim: post-training capability is not one quantity bought with more of one input
• AI Governance — the CAC's five categories, three read
• Anthropic, OpenAI, Google DeepMind — one Recent Activity entry each
• index — a duplicate Astra entry removed. Yesterday's run added the shipped-model line without taking down the "announced 2026-08-01, not released" line beneath it, so the index asserted both for a day