$ cat wiki/papers/2026/2610.28577-cache-engineering.md
Cache Engineering for Agents
TL;DR
AKV lets agents control active KV cache state without rewriting their trajectory or re-prefilling retained context. The authors report lower cost and improved benchmark scores relative to threshold-triggered summarization. (source)
Authors & Org
Tianyu Fu, Ruoxi Wang, Zihan Min, Zhongqi Ren, Zhihao Li, Tengxuan Liu, Shengxu Mu, Hanling Zhang and Yu Wang. First submitted 2026-10-20. Affiliations are not established by the abstract record. (source)
Method
Agents use drop to exclude selected cache entries from future attention and repos to adjust retained keys’ position encodings. Cache state depends on the sequence of operations, so identical token prefixes alone cannot guarantee correct reuse. AKV exposes a stateless serving interface while managing stateful reuse, eviction and restoration internally. The AKV repository is linked by the authors; the full paper and code were not reviewed. (source)
Results
A tool-response rolling-drop policy improves average benchmark score by 1.7 points and reduces average cost by 27.7% against conventional threshold-triggered summarization. With AKV, the authors report up to 1.2× output throughput and a 1.5× speedup in P95 request latency. Recursive improvement of cache policies reduces cost by a further 7.1% over expert design. These are separate author-reported comparisons, not additive savings. (source)
Significance
For Agents (LLM Agents), this makes active attention state an explicit control surface. For Eval Harness Configuration, the history dependence means a reproducible run needs cache-operation semantics as well as token history. The distinction from text summarization is the mechanism that motivates the reported serving benefits. (source)
Open Questions
The abstract does not identify every benchmark, backing model, hardware configuration or policy-search cost. It therefore cannot establish portability of the reported speedups or whether policy optimization pays back at a particular workload scale. Independent replication was not reviewed. (source)