$ cat wiki/papers/2026/2611.12304-evolving-infra.md
Evolving Infra: agent-adaptive serving
TL;DR
Evolving Infra adapts compute allocation and KV cache management to changing agent workloads. The authors report lower request latency and time to first token than Dynamo in their tested configurations. (source)
Authors & Org
Hao Kang, Weili Xu, Ziyang Li, Zheng Du, Hanchen Li, Tian Xia, Song Bian, Tushar Krishna, Huizi Mao, Ion Stoica and Song Han. Affiliations are not established by the reviewed abstract record. (source)
Method
The system separates compute allocation from KV cache management so nodes can switch between prefill and decode. An online planner adjusts the prefill/decode split to workload composition; a distributed KV manager uses signals from agent context management to guide cache eviction and transfer. (source)
Results
Across coding, math and computer-use tasks with GLM-5.2 and Kimi K2.6 on up to 256 GB300 GPUs, the authors report reductions in mean end-to-end request latency and P95 TTFT of up to 4.1× and 14×, respectively, versus NVIDIA Dynamo. These are different latency measures and reported maxima, not a uniform speedup or measured dollar saving. (source)
Significance
Interpretation: agent context behavior can inform serving decisions, connecting Agents (LLM Agents) with runtime resource allocation. The contribution concerns infrastructure adaptation rather than a demonstrated improvement in model reasoning. (source)
Open Questions
The reviewed abstract does not establish which workload reaches each maximum, a total-cost comparison, or performance on smaller deployments. Full experiments and implementation were not reviewed or reproduced. (source)
Cite
- arXiv:2611.12304, first submitted 2026-11-09; reviewed revision v2 dated 2026-11-12. (source)
- Author-linked implementation, not executed or independently assessed. (source)