EN한
← wiki

$ cat wiki/papers/2026/2611.15789-vibeprevisbench.md

VibePrevisBench

paperupdated 2026-11-17created 2026-11-17

TL;DR

VibePrevisBench evaluates whether coding agents can turn a brief into an executable Blender scene and a rendered proxy video. It separates correctness in the scene from whether that correctness is visible in the output. (source)

Authors & Org

Yuzhe Chen, Muxi Diao, Lele Yang, Wenbin Xu, Xuyang Liu, Qile Su, Changyi Liu, Jiankang Chen, Kaiyu Jiang, Haoyu Zhang, Hongyi Fu, Jianxiong Wang, Tianke Zhang, Haonan Fan, Yingxin Li, Bingyi Li, Xu Wang, Xuanyu Zheng, Bin Wen, Tingting Gao, Wentao Zhang and Jiale Cao. Affiliations are not established by the reviewed abstract record. (source)

Method

The benchmark contains 242 tasks with scripts, photos, continuations and reference videos as inputs. Scene Check reruns code and compares the exported 4D scene with hidden facts across nine dimensions. Screen Check uses a VLM judge to test whether the facts that passed are recognizable in the rendered proxy video. The study examines 11 frontier coding agents and their trajectories. (source)

Results

The authors report that spatial and camera-motion understanding distinguishes agents. Basic camera work and staging are generally handled, but challenging requests remain difficult; agents inspect renders without successfully fixing their mistakes. The abstract supplies no per-model scores or uncertainty estimates. These are author-reported observations, not independently reproduced results. (source)

Significance

Interpretation: Eval Harness Configuration should distinguish a valid scene from a visibly correct deliverable. For Agents (LLM Agents), opening a render is not sufficient evidence that its contents were checked against the brief. (source)

Open Questions

How reliable is the VLM screen judge, and how well do the findings transfer to production video workflows? The abstract does not resolve these questions. Full paper, code, videos and generated assets were not reviewed. (source)

Cite

Referenced by

Sources