$ cat wiki/papers/2026/2610.25950-adaexplorebench.md
AdaExploreBench
TL;DR
AdaExploreBench tests whether Agents (LLM Agents) acquire evidence about hidden environmental rules and recognize when those rules change without notice. It distinguishes reasoning from supplied evidence from actively finding useful evidence. (source)
Authors & Org
Rui Wang, Zishen Lai, Jiayu Liu, Cheng Qian, Jeonghwan Kim, Heng Wang, Xuejun Zhang, Jiateng Liu, Bingxuan Li, Shujin Wu, Qing Zong, Tianshi Zheng, Chunyang Li, Qihan Lin, Chen Li, Dadi Guo, Xiusi Chen, Yangqiu Song. Affiliations are not established by the abstract record. First submitted 2026-10-19. (source)
Method
Environmental dynamics are instantiated as hidden rules. Agents must proactively gather evidence and infer them. Under implicit rule drift, rules change without an announcement; degraded performance must trigger recognition that prior knowledge is stale and renewed exploration. (source)
Results
Across 8 leading LLMs, GPT-6 Sol reaches 61.9% calibrated accuracy. Providing sufficient evidence or removing rule drift improves average performance by close to 10%, as stated in the abstract. The abstract does not say whether this denotes relative improvement or percentage points, so no conversion is made. (source)
Significance
Interpretation: the benchmark makes evidence acquisition and detecting obsolete knowledge separate concerns from reasoning once informative evidence is present. This extends the evaluation questions on Eval Harness Configuration. (source)
Open Questions
The abstract does not define calibrated accuracy in detail or provide per-model ablations, interaction budgets or uncertainty estimates. Full paper and code were not reviewed; the result does not establish performance under production distribution shifts. (source)
Cite
“AdaExploreBench: Benchmarking LLM Agents for Proactive Exploration and Test-time Adaptation in Evolving Environments.” arXiv:2610.25950. This note uses the abstract and metadata.