$ cat wiki/papers/2026/2610.28763-agent4sci.md
Agent4Sci: Automating ML for Science
TL;DR
Agent4Sci tests frontier coding agents on improving published scientific ML models. Reported gains mostly come from ML engineering rather than domain insight; research-oriented skills improve both the gains and the diversity of interventions. (source)
Authors & Org
Ming Zhong, Stacy Li, Nicholas Carlini and Matthew Jagielski. First submitted 2026-10-20; the record lists NeurIPS 2026 E&D Track in its comments. Affiliations are not established by the abstract record. (source)
Method
The testbed spans 20 benchmarks across 11 domains and asks how agents improve existing best models under long-running, realistic research conditions. Trajectory analysis examines novelty miscalibration, resource under-utilization and early collapse into local engineering sweeps. The intervention adds skills for research ideation, experiment management and scheduled checklists. The authors link code and data; neither was reviewed, nor was the full paper. (source)
Results
The authors report improvement over the published best on every benchmark across their experiments, with Claude Opus reaching 10.4% mean relative improvement and no run flagged for academic misconduct. Added skills raise Claude Opus to 20.1% and Codex from 4.4% to 8.1%, alongside more diverse, domain-relevant interventions. These percentages are mean relative improvements, not task-success rates or percentage-point gains over an accuracy baseline. (source)
Significance
The distinction between useful optimization and research insight matters for Agents (LLM Agents). The skills intervention also belongs in Eval Harness Configuration: the supplied procedures change what an otherwise capable agent explores. This is evidence about the tested research workflows, not proof of autonomous scientific discovery in general. (source)
Open Questions
The abstract does not specify the exact Claude Opus and Codex versions, per-run budgets, skill-development cost or misconduct detection procedure. No run being flagged is not proof that every run was free of misconduct. The report does not establish independent experimental validation of new scientific findings. (source)