$ cat wiki/papers/2026/2610.31158-omnidr-bench.md
OmniDR-Bench
TL;DR
OmniDR-Bench evaluates video-centered research reports against evidence-linked criteria. The best of 18 agent configurations scores 63.3 out of 100. (source)
Authors & Org
Jiahao Meng, Qize Yang, Yuxuan Wang, Yunfei Chu, Wensi Dai, Jin Xu, Yunhai Tong. Affiliations are not established by the abstract page. First submitted 2026-10-22. (source)
Method
Agents receive a video and user request, gather external multimodal evidence, and produce interactive reports linked to video moments. Omni Atomic Rubric Trees (Omni-ARTs) cover three dimensions, 23 shared directions and task-specific binary criteria. Audited Rubric Construction (ARC) combines independent evidence gathering, automated checks, behavioral calibration and iterative human review. (source)
Results
The benchmark contains 50 Chinese and English tasks, 19 subcategories and 6,414 criteria. Across 18 configurations, the best system scores 63.3 out of 100; the authors identify gaps in coverage, cross-modal causal/logical analysis and citation support. This is a rubric score, not a task-success percentage. (source)
Significance
Agents (LLM Agents) need to connect claims to specific video evidence and external sources. The rubric makes report quality and citation support explicit evaluation targets; Eval Harness Configuration remains relevant because models, skills and harnesses vary. (source)
Open Questions
The abstract does not specify the winning model, total costs or inter-rater agreement. Full paper and code have not been reviewed here; the reported rubric score does not establish deployment reliability. (source)