$ cat briefs/daily/2026-08-15.md
2026-08-15
August 15, 2026 (Sat)
4 stories · 1 paper · 3 watch items · 3 new pages · 1 date closed
Two Chinese labs shipped frontier coding models within hours of each other and split the open-weights deal down the middle: one published the weights and named the licence, the other published the claim and withheld the file. Above both, a quieter item — Anthropic built a benchmark for questions that have no checkable answer, which is the first thing this wiki tracks that cannot be scored the way everything else on it is.
Top Stories
1. A benchmark for the questions that cannot be marked right or wrong
- Anthropic's Alignment Science team published the Conceptual Reasoning Index, combining three benchmarks — LMCA, ACCoRD and DTBench — into one 0-to-100 score for philosophical argument, logical consistency and decision-theory reasoning (source).
- Its target is deliberate: questions practically impossible to verify empirically or mathematically — alignment, governance, collective action under transformative AI — the class where no dataset of past outcomes exists to train on. Built with conceptual researchers Emery Cooper and Caspar Oesterheld.
- Top score as of 2026-08-10: Opus 5 at 73.6. That is the only model figure in anything read here;
alignment.anthropic.comand the project's own site are both blocked from this sandbox, so the rest of the table is unknown to this wiki. - Why it matters: every other benchmark this wiki records a number for measures a task with a checkable answer — does the patch pass, does the exploit land. This one deliberately does not, which raises the question nothing read addresses: how is a benchmark in an unverifiable domain itself validated? And the lab publishing the index also makes the model at the top of it.
- → Conceptual Reasoning Index (CRI) (new), Anthropic, AI Alignment
2. GLM-5.3 is "the strongest open-weights coding model" and you cannot download it
- Z.ai released GLM-5.3 on 2026-08-14 to GLM Coding Plan subscribers from $18/month. Open weights and API access are stated for roughly two weeks out, in stages, after a safety evaluation (source).
- Z.ai states it reuses GLM-5.2's base exactly as it was — every gain from post-training alone, no new pretraining run. Vendor-stated: Terminal-Bench 3.0 4.6 → 28.3, DeepSWE v1.1 66.9, CyberGym 84.5%, and Agents' Last Exam CLI 28.5 against GPT-5.6 Sol's 28.6.
- No per-token price exists. Z.ai's own API pricing table still lists GLM-5.2 and has no GLM-5.3 row.
- For contrast on the same lab: GLM-5.2 put subscriber access and open weights three days apart in June. This is a change of practice, not a lab that always worked this way.
- Why it matters: "open-weights" is being used here to describe a roadmap rather than a file. Read generously, a staged release conditioned on a safety evaluation is the norm arriving in China; read plainly, it is a two-week paid-subscription window with the open label attached. Nothing published separates the two readings, and that ambiguity is new.
- → GLM-5.3 (new), Z.ai, Open-Weights Policy Fight
3. Qwen3.8-27B shipped — four days late, and the countdown was accurate to the hour
- Released 2026-08-14 at 15:00 UTC, which is the 2026-08-15 00:00 JST the ModelScope countdown pointed at. 27.78B dense parameters, text/images/video, Apache 2.0, a 262,144-token native context reported extensible to ~1M via YaRN, on Hugging Face and ModelScope (source).
- Vendor-stated against Qwen3.6-27B: Terminal-Bench 2.1 63.4 → 73.0, DeepSWE 1.1 13.3 → 42.2, OSWorld-Verified 63.9 → 84.3, SWE-MM 25.7 → 38.6, plus SWE-Bench Pro 61.7% and CoWorkBench 70.7% — the last of which Alibaba places above Opus 4.6 Max at 68.2%.
- Yesterday's correction is vindicated. This brief said on 2026-08-14 that the 27B had not shipped, after wrongly recording it as released on 08-12. It shipped two days after the date the wiki had claimed, from a different repository, under a licence nobody had published at the time.
- Alibaba never restated, moved or withdrew the 2026-08-10 date in any first-party channel read here — not before the slip, not on shipping.
- Why it matters: every
unknownon that page for eleven days now has a value, and the one that mattered was License. This wiki declined to assume Apache 2.0 from Qwen precedent; the precedent held, and declining to assume it is still the right rule — the sibling flagship's licence is still unnamed. - → Qwen 3.8 27B, Alibaba / Qwen AI Lab, Open-Weights Policy Fight
4. Auto mode landed on schedule, and the number nobody quoted is the sharper one
- Auto mode is now the default permission mode in Claude Code for Pro, Max and Team, on the date announced 2026-08-07 (source). The 89% / 13.6% comparison, the 1,053 testers and the reversion rules were already ingested eight days ago and are not repeated here.
- Two figures are new. Head to head, the classifier blocked 800 commands the human testers approved, while humans blocked 6 the classifier allowed — a 133-to-1 disagreement. And Anthropic states it will not charge for the tokens the classifier consumes.
- Why it matters: 89% against 13.6% is two independent rates, and invites the objection that the tasks differed. 800 against 6 is the same commands judged by both, which is the comparison that actually settles it. What is still missing is the other half — no false-positive rate, no statement of who labelled "dangerous". 89% is recall, and 800-to-6 says nothing about how many of the 800 were wrong to block.
- → Agents (LLM Agents), Anthropic
Scoring note: story 1 leads at 2.33 — alignment/reasoning (1.3) × Anthropic (1.3) × official-blog signal, plus the new-concept-page bonus, and it is on the tracked-signals list as a new reasoning method from a frontier lab. Story 2 scores 2.10 (agents 1.5, new page), story 3 1.96 (frontier 1.3, weights released +0.4), story 4 1.85 — agents 1.5 × Anthropic 1.3, less the "already well-tracked" deduction, because most of this story was published eight days ago and only two figures are new.
Paper Picks
AI4AI at Test-Time: Strong-to-Weak Capability Transfer via Harnesses (arXiv:2608.12307) — the scaffolding can be written by another model (arXiv:2608.12307, Salesforce AI Research / UIUC)
- Strong-to-weak capability transfer at test time. A stronger builder model constructs an inference-time harness for a weaker target, refining it over multiple rounds against 5% of the data held out as validation. Average across four Theory-of-Mind benchmarks rises 0.49 → 0.91 — with no parameter updates at all (source).
- Why read it: this wiki has spent two weeks recording missing harness disclosure as a reproducibility defect. This measures the same variable from the other end — if a harness can carry a model that far, the unpublished harness in a vendor's benchmark row is not a footnote, it is potentially most of the score.
- Not read.
arxiv.orgis blocked and the HuggingFace snapshot carries no abstract for this entry — only the top-ranked paper has one. Which models, which four benchmarks and any dispersion around that average are absent from the page rather than guessed. - This is the paper yesterday's brief declined to ingest because it then carried no arXiv identifier. It now carries one.
Watch
- GLM-5.3's weights are due around 2026-08-28 — the first entry on Open-Weights Policy Fight that is a promise rather than a measurement. Every other window on that page was timed after the fact: Qwen3.8-27B ran eleven days announcement-to-artefact, Qwen 3.8 Max twenty-four. This one is stated, conditioned on a safety evaluation, and not yet elapsed. → GLM-5.3
- An r/LocalLLaMA thread 72 minutes after release claims Qwen3.8-27B is identical to Qwen3.6-27B. A targeted search found nothing behind it — no reporting, no weight comparison, no repository issue — and it sits directly against the model card's own four-benchmark improvement table. Recorded on the page as unresolved, because an unverified community claim and an unharnessed vendor table are both unchecked, and no third party has measured this checkpoint at all. → Qwen 3.8 27B
- Hugging Face published State of Open Models: Summer 2026 Observations and this run could not read it. Squarely on-topic, and not ingested:
huggingface.cois blocked and a targeted search returned only adjacent material — a Spring 2026 edition, a CNBC interview, a TechCrunch piece. Attributing those figures to a post nobody read is the failure this pipeline exists to prevent. Carried to Sunday's lint, alongside the three Alignment Science back-catalogue articles still uncaptured from yesterday. → Open-Weights Policy Fight
New in Wiki
- Conceptual Reasoning Index (CRI) (new concept page — please review) — the only new page this run that requires user review. Written as a concept rather than filed under an existing page because it has enough published material to fill the schema without padding: a definition, three named sub-benchmarks, a dated score, and two open problems that are specific rather than decorative.
- GLM-5.3 (new model page) — full spec row set.
Context windowisunknownon purpose: Z.ai says the base is GLM-5.2's, unchanged, and that model carries a 1M window — but a shared base is not a published specification, and inferring the row from a sibling is whatunknownexists to prevent. - AI4AI at Test-Time: Strong-to-Weak Capability Transfer via Harnesses (arXiv:2608.12307) (new paper page) — see Paper Picks.
Updates
- Qwen 3.8 27B — released.
Releasednot yet → 2026-08-14,Context window→ 262,144,License→ Apache 2.0,Availability→ Hugging Face + ModelScope. The date table gains its fifth and final row and the withdrawn 08-12 claim stays visible. One harness is disclosed and it is a competitor's: SWE-MM was run on the Claude Code harness, public dev split of SWE-bench Multimodal, modifications per Appendix 8.3 of the Claude Opus 4.7 system card — more harness detail than any release captured this week published for any figure, covering one row of four. - Open-Weights Policy Fight — the two releases recorded as one natural experiment: same day, same country, same claim, opposite artefacts. The 2026-08-12 entry's open question about Qwen's licence is now answered for the 27B and still open for the flagship.
- Eval Harness Configuration — the Qwen disclosure and the AI4AI result recorded together, plus a comparability trap worth naming: Terminal-Bench 3.0 (GLM-5.3) and Terminal-Bench 2.1 (Qwen, DeepSeek) are different suites, three releases in one week, and nothing read maps one onto the other.
- Anthropic — a conflict recorded rather than resolved: Claude for Government has two beta-launch dates 18 days apart. This wiki holds 2026-07-27 from releasebot.io, flagged ⚠️ at the time because no primary page could be confirmed; coverage dated 2026-08-14 reports the beta starting "today" and cites a first-party Anthropic page. Both stand. The ⚠️ flag did exactly what it was put there for.
- Z.ai, Alibaba / Qwen AI Lab, Agents (LLM Agents), AI Alignment — Recent Activity, model lists and cross-links updated.