$ cat wiki/papers/2026/2610.34735-iprefbench.md
iPrefBench: knowing when to infer and when to ask
TL;DR
iPrefBench evaluates personalized mobile agents on both task completion and necessary user interaction. High action completion does not imply that the agent fulfilled the particular user's requirements. (source)
Authors & Org
Yuting Ning, Ruijia Cheng, Sushobhan Ghosh, Peitong Duan, HungShi Lin, Abhishek Sundararajan, Shreyash Pandey, Huan Sun, Jeffrey Nichols and Eldon Schoop. Affiliations are not established by the captured abstract page. First submitted 2026-10-26. (source)
Method
An executable iOS environment pairs an interactive simulated user with a coherent persona containing 3,051 past activity events. It covers 130 underspecified tasks across 26 apps. Each omitted user-dependent requirement is explicitly either inferable from history or obtainable only by asking; evaluation measures personalized completion and user-agent interaction. (source)
Results
Across 7 frontier agents, reported action completion reaches 91.6%, but full task success reaches only 47.7%. These are maxima for different metrics, not necessarily the same agent's paired scores. Agents especially fail to request necessary ask-only information. The results are author-reported in the abstract. (source)
Significance
For Agents (LLM Agents) and Eval Harness Configuration, personalization is part of success rather than a separate style preference. The construction distinguishes evidence available in history from information that requires the user's answer. (source)
Open Questions
How representative is one constructed persona of real users, and do results generalize beyond the simulated-user setting? The abstract does not establish either. Full paper, data and code were not reviewed. (source)
Cite
iPrefBench: A User-Centric Benchmark for Personalized Mobile Use Agents, arXiv:2610.34735, 2026. (source)