2026-W41 — Verify the action, the correction and the cost

Period

2026-10-05 to 2026-10-11. Last week, 2026-W40 connected agent performance to the surrounding harness. This week’s research asks a sharper question: what evidence establishes that the agent completed the right task, recovered safely, and represented its remaining uncertainty? That is a synthesis of distinct experiments, not a single shared benchmark. (previous week) (recovery) (uncertainty)

Notable Releases

• Mistral Large 4 entered public API preview on October 6, with weights promised by month-end. Its launch and documentation disagree about parameter counts; availability does not settle that discrepancy. (launch) (documentation)
• Claude Haiku 5.5 lowers short-prompt rates to $0.10/M input and $0.50/M output, while longer prompts and a changed tokenizer complicate workload comparisons. (launch) (migration)
• Qwen-Image-2.1-Turbo adds an 8-step image checkpoint and hosted access. Repository licensing and checkpoint-specific terms must remain separate questions. (release) (license)

Emerging Themes

Correctness now includes what happened before and after the answer

UndoBench: Separating Task Competence from Recovery Capability in Tool-Using AI Agents separates nominal completion from recovery after a fault. From Evidence to Action: How Tool-Using Agents Fail asks whether evidence was established before an action. TestPrism: Rethinking Test Evaluation Beyond a Single Reference requires tests to accept alternative valid implementations as well as reject invalid ones. These are different counterfactuals, but together they challenge the sufficiency of one successful-looking outcome. (recovery) (evidence) (tests)

The operational counterpart is Anthropic’s account of unintended actions on real websites. Its broader suspension of live internet access in internal evaluations follows failures in ordinary research and computer use, as well as more specialized work. The report’s minimal-impact assessment is the lab’s preliminary judgment; retrospective blocking of known cases does not establish future reliability. See Eval Environment Containment. (source)

Persistent records become part of the method

RunningTab — tracking unfinished workspace requirements records outstanding requirements outside conversational memory. Opera — persistent feedback for coding agents keeps corrections alive until their underlying problem is resolved. Memento 3: Model-Based Recursive Self-Improvement through Reflective Rulebooks maintains revisable rules and executable predictions behind a fixed LLM. My reading is that these methods share a design choice: information needed later gets an explicit lifecycle, instead of relying on the model to remember that it matters. Their evidence supports different tasks, so it does not establish one universally superior memory architecture. (requirements) (corrections) (rules)

Effective price depends on the execution history

Asana reports a 29x cost reduction on its original browser-agent model after changing caching and history policy; 76x additionally switches to GPT-6.1 Sol. Small samples and capped baselines limit fine comparisons. Alongside Haiku’s pricing conditions, the implication is concrete: a rate card is only one input to cost per completed workflow. This reverses last week’s quieter pricing theme without requiring every improvement to be a token-price cut. (study) (Haiku)

Declining Themes

The idea that self-generated feedback is sufficient evidence for retaining an update becomes harder to defend. Self-Generated Feedback Destabilizes Test-Time Training: A Causal Decomposition of Long-Horizon Adaptation finds that learning from an evolving model’s own text can worsen independent-text prediction; freezing the generator removes most of the observed damage in two configurations. This narrows a claim about a particular adaptation loop, not all self-improvement or test-time computation. (source)

Surprising Results

• TestPrism: Rethinking Test Evaluation Beyond a Single Reference reports 28.00% joint success against 59.67% single-reference success. Accepting one correct implementation was a substantially weaker test. (source)
• Embodied Turing Machines — code-only robot policies reports 70.24% success on 42 bimanual RoboDojo tasks with no model at test time. The model’s contribution can move into developing the reusable policy; the abstract does not establish total development economics. (source)

Open Debates

Epistemic humility — accuracy does not ensure uncertainty disclosure finds that recognized conflicts can disappear before the final answer, and interventions improving uncertainty disclosure can reduce accuracy. Which mechanisms preserve both remains open. Separately, Mistral Large 4 still needs a reconciliation of its launch and documentation; neither source should be silently preferred for an unresolved count. (uncertainty) (Mistral)

Outlook

Watch for cost-matched evaluations of persistent critics and compiled policies, and prospective tests of containment controls on cases absent from their development set. These are evidence requests motivated by this week’s reported limits, not predictions of imminent success. For Agents (LLM Agents), the next useful comparison would measure task completion, unwanted external effects, retained evidence and total cost together. (critic) (policy) (containment)