$ cat wiki/papers/2026/2610.32253-rsihub.md
RSIHub: comparing agent evolution under fixed evaluation
TL;DR
RSIHub separates editable agent resources from evaluation machinery. Its largest reported overall pass@1 gain is 13.5 percentage points, but rankings depend on the setting and improvements do not always transfer to held-out tasks. (source)
Authors & Org
Zimu Wang, Xiaobo Wang, Siyu Ye, Jiaqi Li, Silin Chen, Yuling Shi, Ruobing Wang, Jingjing Zhang, Chaofan Wang, Kai Wu, Mengnan Qi, Kai Cai, Xiaodong Gu and Zilong Zheng. Affiliations are not established by the captured abstract page. First submitted 2026-10-23; the metadata names the NeurIPS 2026 Meta-Agents Workshop. (source)
Method
Recipes combine reusable operators and declare which resources may change. Shared infrastructure handles execution, evaluation, recovery and lineage, linking results to candidates and evaluation conditions. The authors implement AHE, Hyperagents, A-Evolve and GEPA without changing the framework core, then test MiniSWE and Codex on Terminal Bench 2 and τ³ Banking with fixed models and evaluators. (source)
Results
The abstract reports the maximum gain above but gives no full score matrix or total optimization cost. It explicitly cautions that gains may not generalize to held-out tasks; no single method is established as universally best. These are author-reported results, not a replication. (source)
Significance
For Agents (LLM Agents) and Eval Harness Configuration, the useful distinction is between evolving the solution and changing its test. Read alongside EvalResearchBench, which studies building the evaluator itself; RSIHub instead fixes it while comparing evolution policies. (RSIHub) (EvalResearchBench)
Open Questions
How much compute does each method spend, and which settings preserve held-out gains? The abstract does not answer either question. Full paper and code were not reviewed. (source)
Cite
RSIHub: A Composable Framework for Agent-Evolution Methods, arXiv:2610.32253, 2026. (snapshot)