BudgetMix RL — testing gains across interaction budgets

TL;DR

The authors find that multi-turn agentic RL gains depend on the allowed interaction budget, with strengths concentrated in recovery and long-horizon resilience. BudgetMix RL varies that budget during training. These are preprint findings, not independent replication. (source)

Authors & Org

Kejian Zhu, Zhuoran Jin, Shangqing Tu, Tianyi Men, Yupu Hao, Jiachun Li, Kang Liu and Jun Zhao. Affiliations are not established by the reviewed abstract page. First submitted 2026-11-04. (source)

Method

• Evaluate sampling breadth K and interaction depth T jointly using Pass@(K,T), exposing sensitivity that a single turn budget can hide. (source)
• Vary interaction budgets during training with BudgetMix RL. The authors also give a preliminary discussion of the benefits and limitations of process supervision. (source)

Results

Trajectory-level RL advantages shrink when turn budgets tighten. Improvements do not uniformly extend to single-step decisions; the authors instead emphasize error recovery and resilience, and report robust BudgetMix RL performance across execution settings. The abstract gives no numerical effect sizes. (source)

Significance

For Agentic Reinforcement Learning and Eval Harness Configuration, training and deployment budgets are part of the comparison. Interpretation: a reported agent gain should be checked across the interaction limits the deployed system will actually face. (source)

Open Questions

The reviewed abstract does not quantify training overhead or establish how broadly the results transfer across tasks and model sizes. Full-paper methods, code and independent reproduction remain unreviewed here. (source)

Cite

What Does Multi-Turn Reinforcement Learning Really Improve in LLM Agents?

arXiv:2611.06483, first submitted 2026-11-04. Project page. (source)