Memory as Plans: World-Action Modeling with Memory-Grounded Planning

TL;DR

Robot policies are mostly Markovian and many real manipulation tasks are not. MaP-WAM splits memory-dependent control into memory-grounded planning and plan-conditioned execution, using long-term multimodal episodic context as planning-time evidence instead of feeding the executor the whole history. 83.3% success on RMBench and 78.0% on real-robot tasks, with executor inference latency "approximately constant" as the history grows (source).

Authors & Org

Not stated — the HuggingFace Daily Papers snapshot carries id, title, date, upvotes and abstract only. Not read first-party.

Method

The stated problem with existing memory mechanisms — language summaries, growing visual windows, or both — is that they either "lose fine-grained visual evidence" or face "a trade-off between history coverage and execution efficiency."

• Memory as completed segment records: each holds a language instruction plus sparse visual context, rather than a full frame history.
• Compact plans: episodic memory is converted into the next segment-level language plan and its corresponding visual guidance.
• World-Action-Progress (WAP) model: executes a plan over an unknown duration by jointly predicting action chunks and execution progress at inference time, calibrating that progress through plan-observation alignment for adaptive segment transitions and closed-loop context updates.
• Fixed executor context length, with structured attention enabling key-value caching in both planning and execution.

Results

Claim	Figure
RMBench success rate	83.3% (stated state-of-the-art)
Real-robot tasks	78.0% success
Executor inference latency vs. task history	approximately constant
**No baseline figures, no ablation numbers and no task counts appear in the
abstract**, so the state-of-the-art claim is reported and the margin is not.

Significance

The interesting move is architectural, not numeric: history is spent at planning time and the executor's context stays fixed, so the cost of remembering does not ride on every action step. That is the same pressure Agents (LLM Agents) records for long-horizon software agents — context growth degrading the loop that has to run most often — solved by moving the growth to the part that runs least often.

The constant-latency claim is the one to check, because it is the property that would make the approach deployable, and it is exactly the property the abstract supports with an adverb ("approximately") and no measurement. → Embodied Agents

Open Questions

• Baselines absent. 83.3% against what? RMBench numbers for the language- summary and growing-window mechanisms it argues against are not given.
• What is the cost of a wrong plan? Segment-level plans over an unknown duration mean a bad segment boundary is a failure the executor cannot see; the abstract states calibration exists and not how often it is needed.
• Real-robot task set undescribed — 78.0% over an unstated number of tasks on unstated hardware.
• Author and affiliation unknown, so no org weight was applied.

Cite

• arXiv: 2609.11561
• Surfaced via HuggingFace Daily Papers, 2026-09-14, 36 upvotes — a popularity signal from that community, not a quality ranking (source)