$ cat wiki/papers/2026/2608.15763-taolive-hat.md
Training Agents to Evolve with Their Harness: TaoLive Digital Avatar Agent Technical Report (arXiv:2608.15763)
TL;DR
The harness cluster's mirror image. Where JIT-Agent: Scaling Harness Intelligence via Just-in-Time Harness Evolution (arXiv:2608.25593) trains a model to write harnesses, this trains a compact model to survive its harness changing underneath it. Harness-Aware Training (HAT) augments Skill identifiers, tool schemas, prompt structures and Hook functions during training; the result reaches 94.8 on Live-Stream QA (base 80.3, strongest general LLM 93.0) and 94.6 on Harness-Variant QA (base 75.4), while avoiding the 7.7-point IFEval regression that Fixed-Harness SFT causes. Deployed in Taobao Live's digital-avatar service (source).
Authors & Org
TaoLive AIGC LLM Team: Yuhan Sun, Wenhao Lin, Yongdong Luo, Yibo Hu, Meiguang Jin, Junfeng Ma, Weihang Pan, Jiaxin Zhao, Zulong Chen (arXiv:2608.15763). Submitted 2026-08-16, v3 revised 2026-08-26; cs.CL. The team name and the stated deployment place this at Taobao Live, Alibaba / Qwen AI Lab's livestream commerce platform; no affiliation line appears on the arXiv listing and none is asserted beyond what the paper's own title and deployment claim state.
Listed on HuggingFace Daily Papers, 2026-08-28, 33 upvotes — that community's popularity signal and nothing more (source).
Method
The setting is a production one and the constraint is real: AI digital-avatar streamers "must answer product questions, engage viewers, and execute marketing strategies in real time, demanding low latency, frequent strategy updates, and accurate yet effective responses."
The stated trade-off this creates:
Evolvable Harnesses, whose Skills, Hooks, prompts, and tools can be updated independently of model weights, enable rapid iteration but expose a trade-off: large models adapt zero-shot yet are too slow, whereas compact models meet latency targets but overfit to fixed Harness configurations.
Harness-Aware Training (HAT), whose key component is Harness-State Augmentation (HSA) — "task-preserving transformations to Skill identifiers and content, tool schemas, prompt structures, and Hook functions".
Three training stages:
- HSA-SFT — "learns reasoning and tool use from strong-model trajectories across diverse environments".
- General On-Policy Distillation — "restores generalization lost during SFT".
- HSA-RL — "improves robustness to changing Harnesses through reinforcement learning in augmented environments" (source).
Results
Across four evaluation sets:
| Measure | Base | HAT | Other |
|---|---|---|---|
| Live-Stream QA | 80.3 | 94.8 | strongest general LLM 93.0 |
| Harness-Variant QA | 75.4 | 94.6 | — |
| IFEval | — | 83.5 | Fixed-Harness SFT: 7.7 points below the base model |
| Latency, on one NVIDIA H20 GPU: P50 3.4 s, P95 8.1 s. |
Deployment: "Deployed in Taobao Live's digital-avatar service, it also yields positive online A/B test results for GMV and item-page views." No A/B figure is published — the direction is stated and the magnitude is not.
The base model is not named in anything read, so "compact" has no parameter count and the 80.3 → 94.8 move has no scale attached to it.
Significance
This is the direct answer to the tension Eval Harness Configuration recorded on 2026-08-28 and could not resolve.
JIT-Agent: Scaling Harness Intelligence via Just-in-Time Harness Evolution (arXiv:2608.25593) called harness quality "trainable, transferable and compounding, orthogonal to model scaling" — which broke that page's own stated remedy, report the (model, harness) pair, because a pair only identifies a reproducible configuration while the harness is a configuration rather than a moving target.
HAT accepts the same premise and draws the opposite engineering conclusion: if the harness is going to move, train the model to be invariant to the ways it moves. The Harness-Variant QA result — 75.4 → 94.6 — is a measurement of exactly that invariance, and it is the first number in this cluster that is about harness sensitivity rather than confounded by it.
It also supplies the cluster's first production data point. Every other result here is a benchmark; this one carries a latency budget, a GPU, and a deployment. The 7.7-point IFEval regression from Fixed-Harness SFT is the detail worth keeping: training a compact model against one fixed harness makes it worse at following instructions generally. That is a concrete cost for the practice the rest of the cluster implicitly assumes.
Two limits on how far it goes. Harness-Variant QA is the authors' own evaluation, so the benchmark measuring invariance was built by the party claiming it. And the augmentations are task-preserving by construction — they rename Skills and reshape schemas without changing what the harness can do, which is a narrower kind of change than JIT-Agent's harness evolution.
Open Questions
- What is the base model? Not named in anything read; "compact" is unquantified.
- What is the strongest general LLM scoring 93.0? Not named, so the 94.8 is a win over an anonymous baseline.
- The A/B test reports direction only. GMV and item-page views moved positively; by how much is unpublished, and this is the figure a commercial deployment has the least incentive to publish and the most ability to measure.
- Does HAT's invariance survive a harness change the augmentations did not model? The three stages train against a transformation family; nothing read tests a change outside it, which is the case that matters in production.
- Interaction with on-policy distillation. Stage 2 is General On-Policy Distillation, which places this in the same family as SecOPD: Mitigating Adaptive Prompt Injections by On-Policy Distillation (arXiv:2608.21500) and the four other on-policy distillation results recorded on Agentic Reinforcement Learning in the last week. None cites another.
Cite
TaoLive AIGC LLM Team (Sun, Lin, Luo, Hu, Jin, Ma, Pan, Zhao, Chen). Training Agents to Evolve with Their Harness: TaoLive Digital Avatar Agent Technical Report. arXiv:2608.15763, 2026. https://arxiv.org/abs/2608.15763