Memory Tax — the net value of agent memory

TL;DR

The authors compare dedicated agent memory with direct retrieval from raw history and report that many memory systems underperform naive RAG in the evaluated settings. They call the losses caused by transforming and accessing history a memory tax; this does not establish that every memory system is harmful. (source)

Authors & Org

Xuan He, Sining Zhoubian, Ruozhen Yang, Yuhao Wen, Zhineng Wang, Yuyang Hu, Ke Yang, Yuyu Zhang, Ping Nie, Jiawei Han and ChengXiang Zhai. Affiliations are not established in the captured abstract record. First submitted 2026-10-25. (source)

Method

Using RAG as the reference, the study separates memory benefit from memory tax across multiple benchmarks and model backbones. Reorganizing facts or distilling preferences can help access, but can also remove or obscure evidence. The authors examine both system design and evaluation as sources of this loss, then extend the framing to other memory applications. (source)

Results

The abstract reports frequent underperformance against naive RAG without naming the full benchmark/model matrix or providing numerical effect sizes. It promises to release code and results; artifact availability was not verified in this review. (source)

Significance

For Agents (LLM Agents) and Context Compaction, the useful comparison is the net gain over preserved raw information. This complements Raw-RAG — raw-history retrieval as a memory baseline: a memory mechanism needs to justify the evidence it changes, not just its standalone score. (source) (earlier study)

Open Questions

The abstract leaves model settings, retrieval budgets, costs and individual failure rates unspecified. Full-paper review is needed to separate information loss from retrieval and evaluation effects. No independent replication was performed. (source)

Cite

• arXiv:2610.34073 — first submission 2026-10-25.
• Captured abstract.