$ cat briefs/daily/2026-08-21.md
2026-08-21
August 21, 2026 (Fri)
8 papers · 1 new concept · 5 stories · 2 paper picks · 3 watch items
Top Stories
1. The harness that trains a model finally gets named — and the spread is 6.8 points before training starts
- LEGO-RL: Harness-Native Reinforcement Learning for Coding Agents (arXiv:2608.17393) trains Qwen3.5-35B-A3B with GSPO inside three unmodified production harnesses and names all three: OpenHands SDK 64.0% → 70.4%, Claude Code 62.4% → 68.2%, OpenCode 57.2% → 66.6% on SWE-bench Verified (HF Daily 2026-08-21)
- This wiki's sharpest objection to yesterday's Agent Lightning v1.0: Towards Harnessed Agentic RL (arXiv:2608.17528) was that the paper arguing hardest for the harness did not name the one behind its headline number. A different group supplied it one day later
- The untrained starting points differ by 6.8 points from harness alone — larger than two of the three trained gains
- It also reports rollout–training probability correlation above 0.99, the quantity a harness destroys when it compacts context mid-rollout. No benchmark delta reveals a broken correlation, and only one of the three harness-training papers here reports it
- Why it matters: a model post-trained through a harness is fitted to it, so "harness-agnostic" becomes something to demonstrate rather than assume — and the integrity check says whether the training was even valid
- → Eval Harness Configuration, Agentic Reinforcement Learning
2. Anthropic is reported to be unwinding the retention mandate this wiki recorded yesterday
- Bloomberg, 2026-08-20, citing an unnamed source: enterprise customers still required to retain 30 days, but gaining the option to hold the data on their own cloud infrastructure; rollout later this year, developed with 100+ customers including Salesforce (source)
- Safety Monitoring and Data Retention argued yesterday that Anthropic's position was the sharper of the two because it was enforced while OpenAI's was a preview. Both are plans now, and Anthropic's lands on customer-controlled infrastructure — the phrase in OpenAI's own 08-19 post
- Sourcing is the weakest tier this repo accepts: third-party reporting of an intention, no first-party statement surfaced, June's policy still in force
- Why it matters: the disagreement narrows from "must content be retained" to what the lab receives when the customer holds it — and neither lab has said. If Anthropic must still compute over the content, this moves residency and liability but not capability
- → Safety Monitoring and Data Retention, Anthropic
3. A lab CEO asserts post-training scaling as a law, with his own model as the experiment
- Z.ai's Jie Tang, reported 2026-08-20: memorization prefers parameters; reasoning prefers post-training data and effective depth; advanced skills do not live in parameter count past a knowledge threshold. Five scaling knobs, an "XA-YB" MoE notation (source)
- GLM-5.3 is the evidence: GLM-5.2's base untouched, all gains from RL on long-horizon environments — Terminal-Bench 3.0 4.6 → 28.3, DeepSWE 46.2 → 66.9, AutomationBench 26.2 → 48.2
- The first two match this repo's independent 2026-08-14 capture, which is why the new figures were accepted into the model page rather than filed as a competing account. All of it vendor-stated, no harness published for any figure
- Why it matters: Weekly Synthesis — W33 (2026-08-10 → 2026-08-16) read "capability stopped arriving in the weights" off four release notes; this is the first time someone shipping models has claimed it as a law — and if the base is the cheap half, publishing weights transfers less than Open-Weights Policy Fight has assumed
- → Post-Training Scaling, Z.ai
4. A benchmark everything passes equally turns out not to be measuring the thing
- SemaPLC: A Project-Grounded, Verification-Gated Agent Harness for PLC Code Generation (arXiv:2608.18565) gates task completion on logged external checks, never the model's own judgement — specification, compilation, and executed traces from a live PLC runtime compared against a reference (HF Daily 2026-08-21)
- Static behaviour scores put every method within 10 points of one another. Dynamic behaviour spreads them from 22.4–31.4 up to 52.2
- 72.6% mean strict verified pass rate, highest on all seven models tested
- Why it matters: take away the PLCs and the finding is a general one about instruments — and its limit is honest, since verification-gating only works where verification is cheap, which is where agents were already most reliable
- → Agents (LLM Agents), Eval Harness Configuration
5. The freeze-the-model pattern crosses into physical control, for a different reason
- Zetta ζ: An Efficient Closed-Loop Embodied Harness for Self-Evolving Physical Intelligence (arXiv:2608.16590) evolves code-based runtime critics and recovery skills online with the base policy frozen: LIBERO-Pro 90.8%, RoboCasa 93.6%, 11.1× inference speedup (HF Daily 2026-08-21)
- Both success figures are scoped to an unpublished rollout budget — the paper's own qualifier, so neither is comparable to another system's
- Why it matters: DarwinX: Evolving Agent Harnesses Through Natural Selection (arXiv:2608.07545) froze the model because harness evolution paid better; Zetta freezes it because physical interaction needs decisions faster than a large agentic model can make them. That is an argument from frequency, not headroom, and it does not transfer from the software cluster
- → Embodied Agents
Paper Picks
Training Leaves Traces: Centered Residual Signatures for Language Model Lineage Verification — arXiv:2608.14929
- TL;DR: a data-free, white-box test for shared ancestry read from weights alone — AUROC = 1.0 separating fine-tuned, LoRA-merged, pruned and quantized descendants from independent models, unchanged under function-preserving laundering, 76× faster than the nearest robust baseline
- Why read it: distilled models group with the independent ones. It measures weight ancestry, not behavioral similarity, so it audits licence compliance without the publisher's cooperation and cannot detect distillation at all — the opposite of what the headline number suggests
- → Training Leaves Traces: Centered Residual Signatures for Language Model Lineage Verification (arXiv:2608.14929)
Co-RL: Unsupervised Reasoning Emerges from Diverse Cohort in Multi-agent RL — arXiv:2608.17253
- TL;DR: several parameter-independent models trained simultaneously, each rewarded by its peers; 3.0–8.6% across seven text benchmarks and 2.3–7.2% across four multimodal ones, with no ground-truth labels
- Why read it: it reframes self-rewarding RL's collapse as an error-correlation problem rather than a bad-judge one, which turns cohort diversity from a heuristic into a mechanism with a testable boundary
- → Co-RL: Unsupervised Reasoning Emerges from Diverse Cohort in Multi-agent RL (arXiv:2608.17253)
Watch
- OpenAI stands up a Strategic Futures team with a constitutional remit — Introducing AI Futures, 2026-08-20, reported to ask how a free society should be restructured under transformative AI. Every other OpenAI governance artefact here is tied to a capability and a framework; this one is not. Leadership, staffing and cadence all unstated (source) → AI Governance
- "Pick sides" acquires a mechanism. A State Department letter is reported drafted to the 35 signatories of June's AI Opportunity Partnership statement; MoFA's Lin Jian rejects bloc confrontation. Nothing read establishes the letter was sent (source) → AI Governance
- Liquid AI names a model this wiki cannot see. LFM2.5-DSpark speculative decoding claims up to 3.18× on an H100 with outputs unchanged, and lists LFM2.5-8B-A1B as a target — no parameters, context window, licence or release date anywhere read. No draft-block acceptance rate published either, which is what the speedup rests on (source) → Liquid AI
New in Wiki
- Post-Training Scaling (new — concept page, needs your review: it holds a vendor's claim and five independent papers side by side, deliberately separated)
- LEGO-RL: Harness-Native Reinforcement Learning for Coding Agents (arXiv:2608.17393) · Zetta ζ: An Efficient Closed-Loop Embodied Harness for Self-Evolving Physical Intelligence (arXiv:2608.16590) · SPADE: Self-Play in Adaptive Synthetic Executable Environments (arXiv:2608.19197) · Training Leaves Traces: Centered Residual Signatures for Language Model Lineage Verification (arXiv:2608.14929) · Co-RL: Unsupervised Reasoning Emerges from Diverse Cohort in Multi-agent RL (arXiv:2608.17253) · SemaPLC: A Project-Grounded, Verification-Gated Agent Harness for PLC Code Generation (arXiv:2608.18565) · OmniScientist: An Omni-Modal Omni-Discipline AI Scientist (arXiv:2608.13558) · Looped Language Models Improve Compositional Tool Calling (arXiv:2608.18171)
Updates
- Eval Harness Configuration: new dated section — the harnesses named, and the 0.99 correlation check
- Agentic Reinforcement Learning: LEGO-RL, Co-RL and SPADE added; reward-hacking problem sharpened (SPADE's environments write their own verifiers)
- Open-Weights Policy Fight: two new open problems — whether open weights still transfer capability, and a provenance test that audits licences but not distillation
- Embodied Agents: 21 days stale, the run's largest edit — Zetta added
- AI Governance: the Pax Silica letter gets an addressee set and a named alternative (WAICO); OpenAI AI Futures
- Safety Monitoring and Data Retention: the 24-hour inversion, plus a new open problem on custody vs capability
- Test-Time Compute (Inference-Time Compute Scaling): a fifth mechanism — recurrent depth, and adaptive allocation for the third time this month
- Agents (LLM Agents): SemaPLC and OmniScientist
- GLM-5.3: DeepSWE
unknownfilled (46.2), AutomationBench row added - LFM2.5-2.6B, Liquid AI: DSpark and QAD, kept out of the benchmark table on purpose
- Anthropic · OpenAI · Z.ai · Claude Fable 5 — see stories above