FrontierPhysics — evaluating complete physics research

TL;DR

FrontierPhysics tests research planning, execution and manuscripts rather than isolated physics answers. Across the reported configurations, none passes more than 46 of 168 rollouts. These are author-reported preprint results. (source)

Authors & Org

Bingran You and 71 coauthors; the full author list is in the captured arXiv record. Affiliations are not established by the reviewed abstract page. First submitted 2026-11-04. (source)

Method

• 42 physicists construct 56 tasks from their own research across nine theoretical, computational and experimental domains. Agents must review literature, plan and conduct research and experiments, then produce structured outputs and manuscripts. (source)
• Deterministic verifiers and expert-authored rubrics assess correctness and publishability. The evaluation covers 22 models in 26 configurations. (source)

Results

No configuration passes more than 46 of its 168 rollouts. Manual trajectory review attributes the main weaknesses to insufficiently rigorous reasoning and limited expert intuition, rather than knowledge gaps or execution errors. The denominator is rollouts, not distinct tasks or accepted journal papers. (source)

Significance

For Agents (LLM Agents) and Eval Harness Configuration, the useful distinction is between completing a research workflow and meeting its scientific quality criteria. Interpretation: successful tool execution alone cannot establish research success. (source)

Open Questions

The abstract does not identify the best configuration, establish equal compute budgets, or supply independent validation of rubric-based publishability. Full-paper details, code and reproduction were not reviewed here. (source)

Cite

FrontierPhysics: Evaluating Agents for End-to-End Physics Research

arXiv:2611.06910, first submitted 2026-11-04. (source)