Cortex: retaining context gains in model parameters

TL;DR

Cortex distills context-conditioned answers into parameter updates so gains can survive removal of the original context. Its evaluation measures both consolidation and retention of unrelated capabilities. (source)

Authors & Org

Jintian Zhang, Feng He, Xinnian Liang, Ziwen Xu, Haoming Xu, Haojing Huang, Shuangzhi Wu and Ningyu Zhang. Submitted 2026-10-14; affiliations are not established by the captured abstract page. (source)

Method

The method generates prospective queries about how supplied context might be used, then distills the context-conditioned outputs into mergeable low-rank or structural feed-forward updates. A five-task-family testbed covers single, batch and sequential updates, connecting factual editing, skill learning and behavioral steering. (source)

Results

Across 4B--120B dense and mixture-of-experts backbones, including text-only and multimodal models, the authors report an empirical consolidation-retention Pareto frontier in most tested settings. After 50 sequential factual edits on a 14B model, Cortex-Structural recovers 88.9% of the context-induced gain while retaining 99.8% of original non-target performance. These are relative preservation measures, not absolute task accuracies. (source)

Significance

This offers a parameter-based memory route alongside the textual and soft-memory approaches recorded in Context Compaction. The explicit retention objective makes interference with other capabilities part of the evaluation. (source)

Open Questions

How do consolidation cost, reversibility and retention behave over longer sequences of conflicting updates? The abstract does not settle these questions; the full paper and code have not been reviewed here. (source)

Cite

• Cortex: Consolidating Context into Parametric Memory, arXiv:2610.20399.
• Captured abstract page; selected from HuggingFace Daily Papers, 2026-10-16.