$ cat wiki/papers/2026/2610.14146-lhtb-2.md
Long-Horizon-Terminal-Bench 2.0: measuring sustained progress
TL;DR
LHTB 2.0 evaluates progress during hours-long terminal tasks, retaining partial progress that binary completion checks miss. It extends and repairs Long-Horizon-Terminal-Bench (LHTB). (source)
Authors & Org
Zongxia Li, Zhongzhi Li, Yucheng Shi, Ruhan Wang, Junyao Yang, Chengsong Huang, Zhichao Liu, Xiyang Wu, Anhao Li, Yue Yu, Ergan Shang, Fuxiao Liu, Tong Zheng, Ninghao Liu, Lichao Sun, Haitao Mi and Leowei Liang. Submitted 2026-10-09; affiliations were not established from the abstract page. (source)
Method
The benchmark preserves 46 original tasks and extends 31, for 77 tasks across 13 domains and eight task types, while correcting task and infrastructure defects. Reconstruction tasks reveal six reference scenarios and hold out three; dense progress ladders reward intermediate work. Runs extend from 90 minutes to three hours, with verifier checkpoints every 30 minutes. (source)
Results
Across 1,145 scored runs and 15 frontier models, GPT-6-astra leads at 0.69 dense mean reward and 37 tasks with R at least 0.95; GPT-5.6-sol follows at 0.67 and 32. On 30 reconstruction tasks, mean binary scores are 0.221 versus 0.653 for dense ladders. These are author-reported results under this evaluation. (source)
Significance
For Eval Harness Configuration, the scoring rule changes how much progress is visible. A dense reward is not a completion rate, and this version's results should not be directly equated with the earlier benchmark's scores. (source)
Open Questions
The abstract says 31 tasks are extended but reports reconstruction results on 30; it does not explain that subset. Harness settings, budget sensitivity and verifier reliability require full-paper review, which has not been performed here. (source)