MPO — turn-level optimization for long-horizon agents

TL;DR

Multiscale Policy Optimization (MPO) aligns agent policy updates with complete assistant turns while allocating credit more finely among tokens. The authors report gains across search, tool use and automation; the abstract does not quantify the improvement. (source)

Authors & Org

Tao Ren, Hui Yang, Jia Deng, Tingyang Chen, Qiyao Wang, Shuo Tang, Ziyang Zeng, Yuhua Jiang, Guanghao Li, Huatong Song, Yining Li, Yifan Luo, Kunbao Ding, Feng Chang, Chuan Hao, Ran Tao, Bryan Dai, Jiafeng Liang and Jinyang Jiang. Affiliations are not established in the captured abstract record. First submitted 2026-10-26. (source)

Method

The starting observation is that critic values vary mainly across turns even though the model generates action tokens sequentially. MPO combines Multiscale Advantage Estimation (MAE) with turn-level importance sampling, aligning both credit assignment and policy updates with the scale at which the agent interacts with its environment. (source)

Results

Experiments use 4B and 30B-A3B models on search, tool-use and automation tasks. The abstract describes consistent gains over strong RL baselines, but supplies no baseline names, individual scores or uncertainty estimates. (source)

Significance

For Agentic Reinforcement Learning and Agents (LLM Agents), MPO makes the assistant turn an explicit optimization unit rather than treating the entire task as an undifferentiated token stream. The reported motivation concerns where value changes occur; it is not evidence of lower end-to-end training cost. (source)

Open Questions

The full paper is needed to inspect how token credit is allocated, how baselines are matched and whether gains survive different turn lengths and tool latencies. Training cost and independent replication were not established by this abstract review. (source)

Cite

• arXiv:2610.35013 — first submission 2026-10-26.
• Captured abstract.