$ cat briefs/daily/2026-10-06.md
2026-10-06
October 6, 2026 (Tue)
2 stories · 2 paper picks · 2 new pages
Top Stories
1. OpenAI begins a limited text-watermark rollout
API customers can opt in globally for select models; eligible ChatGPT and Codex text in the EU follows over the coming weeks. Detector access initially requires approval. In a 400-token editing test, replacing 10% of words with synonyms reduces detection from about 92% to 66%; replacing 25% reduces it to 17%.
Why it matters: output marking is becoming a deployed feature, but a missed watermark cannot establish human authorship. → Content Provenance (AI output marking) (source)
2. SciUniverse tests whether agents can carry out laboratory work
C5R evaluates 92 tasks across 17 equally weighted families, mixing physical and virtual work. Claude Fable 5.1 at xhigh leads the displayed overall Pass@1 table at 45.3%, versus GPT-6 Astra at xhigh at 32.5%. Reported costs of $40.61 and $52.37 per task cover inference only; laboratory and labor costs are excluded.
Why it matters: a benchmark tied to experimental outcomes is useful evidence, while limited physical rollouts and incomplete costs constrain deployment conclusions. → Eval Harness Configuration (source)
Paper Picks
RealCompanion — when does conversational memory help? The current abstract reports 95.9% recent-message retrieval success overall, but 2.2% when the needed message is far back. The study uses ten real relationships and 27,218 messages. Read it for the distinction between retrieval accuracy and deciding when history matters; the wiki preserves a denominator discrepancy with today's saved abstract. → RealCompanion: understanding people across long conversations (source)
VeriHarness — verify consensus as well as disagreements. A fixed model gets a workspace, evidence tools and reusable verification skills. Evidence-backed revision improves average performance over one rollout by 6.2 points with Gemini 3.5 Flash and 6.4 with Claude Opus 4.8. Read it for a concrete verification mechanism; the abstract does not establish the extra cost per task. → VeriHarness: checking agreement against evidence (source)
Watch
- FrugalEvo: stronger-model planning plus cheaper implementation improves a cost-aware quality metric on 9 of 10 optimization tasks. Promising author-reported evidence for measuring progress over spending, rather than iterations alone. → Agents (LLM Agents) (source)
- Local reasoning settings matter: Simon Willison's Qwen3.8-27B Q4_K_M experiment gets 23.57% arithmetic accuracy over 5,070 reasoning-disabled cases, versus 167 of 169 correct in a medium-reasoning pilot. Different sample sizes limit the comparison. → Qwen 3.8 27B (source)
New in Wiki
- RealCompanion: understanding people across long conversations — memory benchmark note, with source-version limits. (source)
- VeriHarness: checking agreement against evidence — verification method note and open cost question. (source)
Updates
- OpenAI now links to the provenance analysis. (source)
- Eval Harness Configuration records a source-label correction: Import AI calls one entry GPT-5 Astra, while C5R identifies GPT-6 Astra. (newsletter) (benchmark)