EN한
← wiki

$ cat wiki/papers/2026/2611.17167-decision-model-roles.md

Decision models as actors, judges and teachers

paperupdated 2026-11-15created 2026-11-15

TL;DR

A weak actor can still provide useful training feedback after observing a complete trajectory. The authors test decision models in distinct actor, judge and teacher roles. (source)

Authors & Org

Haochen Liu, Qianhao Zhu, Zixuan Wang, Dingming Li, Tongbo Chen, Peng Qiu, Yingqi Qu, Weiming Lu, Jun Xiao and Yongliang Shen. Affiliations are not established by the captured metadata. First submitted 2026-11-12. (source)

Method

Compare three decision models and three general LLMs as actors and judges on ALFWorld; use JEV as a teacher on ALFWorld and WebShop. A judge sees the whole trajectory and evaluates whether an action helped, rather than predicting whether the task will succeed. (source)

Results

All six actors can be distracted from a prerequisite subgoal by an easier one, while retrospective judges credit progress toward it. Teaching a Qwen2.5-1.5B policy with JEV improves success over GRPO from 72.8% to 82.9% on ALFWorld and 56.8% to 71.4% on WebShop, according to the abstract. These are success rates, not relative percentage gains. (source)

Significance

For Agentic Reinforcement Learning, useful supervision need not come from the strongest acting policy: hindsight exposes consequences the actor must anticipate. SO-Arena separately tests reliability beyond aggregate capability, supporting role-specific evaluation without pooling the two studies' results. (source) (reliability)

Open Questions

Full-paper methods and code were not reviewed. The abstract does not establish training cost, transfer beyond the tested environments or whether the result holds for every teacher. (source)

Referenced by

Sources