World Models

Definition

A world model is a learned model of an environment's dynamics: given a state and an action, it predicts what happens next. The term is used in this wiki for two things that share a name and little else, and the distinction is the page's reason to exist:

• Generative world models — a video (and increasingly audio) model made interactive, so the generated stream responds to inputs as it runs. GWM Worlds 2 is this.
• Predictive world models for agents — a model an agent queries before acting, to anticipate the result of a candidate action without paying for it. Agent-Editing World Model: Rethinking World Modeling for LLM Agents is this.

Both were in this wiki's intake on the same day, from unrelated publishers, and neither cites the other's tradition.

Why It Matters

The two readings imply different tests, and conflating them means a release gets credit on the wrong one. A generative world model is judged on whether the stream stays coherent under control; a predictive one is judged on whether acting on its prediction raises task success. A video model that looks right has not thereby been shown to know what happens next, and a state predictor that raises success rates need not render anything at all.

This wiki accumulated world-model material for months without a page holding it: Cosmos-H-Dreams and Qwen-Drive-1.0-4B both carry the term in comparison tables, Embodied Agents uses it, and the two papers below arrived with GWM Worlds 2 in one run. The page is created here rather than left as scattered mentions.

State of the Art (as of 2026-09-27)

The instrument this page said did not exist was published five days before this page was written.

This page was created on 2026-09-26 with the observation that the generative and predictive senses of world model arrived in one run from unrelated publishers and that there was no instrument on which they could be compared. HappyWorld-Bench is dated 2026-09-21 and spans both senses (source):

Track	Sense of the term	Scale	Systems
Video world models	generative coherence	1,138 prompts	14
Spatial world models	placement and editing	300 scenes	9
Embodied world models	state under action	254 test cases	8
Its premise is the one this page argued for: that evaluation needs both the quality
of the world generated and its **consistency and responsiveness under exploration,
interaction and modification**. Structure is a **hierarchical framework of six
capabilities, W1–W6**, from generative construction to unified world modelling — **the
six are counted and never named** in anything read. Measurement runs two ways at once:
HappyWorld-Arena human A/B comparisons yielding model-level Elo, alongside
newly designed automated metrics for behavioural correctness.

The numbers, such as they are: spatial models reach at best 70.14% placement accuracy and 73.33% edit execution — both ceilings, so the other systems are below them. Video models show reduced consistency during extended rollouts and revisits; embodied models struggle to preserve state across multi-step actions and to respond precisely to altered action conditions and physical rules. Only the spatial track is quantified at all.

So the gap was in this pipeline's reading, not in the field. Recorded plainly, because the alternative is a wiki taking credit for noticing an absence that was not there. What remains true is narrower and still worth holding: the two senses are not compared against each other even here — HappyWorld-Bench evaluates them in three independent tracks, which is a shared harness rather than a shared metric.

Against GWM Worlds 2: Runway publishes no benchmark of any kind and concedes that real-time generation "trades fidelity for speed" with no figure on either side. A 1,138-prompt video track with human Elo and behavioural metrics is exactly where that trade becomes a number. Nothing read states GWM Worlds 2 is among the 14, and it could not be — it is contact-only.

Three world-model papers in one week now measure the same property — does the world remember what it did: this benchmark, 2609.28654 Training Object Permanence in World Models (196 upvotes today, the snapshot's highest) and 2609.28466 The Past Frames the Future. The strongest figure in any of them is 73.33%.

State of the Art (as of 2026-09-26)

A world model shipped as a playable stream (2026-09-03)

Runway's GWM Worlds 2 generates continuous 720p video at 24 fps with 48 kHz audio, not restricted to a fixed duration, under an input format Runway calls WorldPrompt — the first frame and other elements are fixed up front, then timestamped events steer characters, cameras and environment, and events can be issued while generation runs. Stated recipe: fine-tune a base video model to the format, post-train for autoregressive generation, distil for real-time speed (source).

No benchmark accompanies it, which is the state of the art's honest summary for this branch: the demonstration is the evidence, and Runway itself states that real-time generation "trades fidelity for speed" without quantifying the trade.

Object permanence is measurable, and video models mostly lack it (2026-09-23)

WROP (Training Object Permanence in World Models, arXiv 2609.28654, 155 upvotes) asks whether video generation models have emerged object permanence and solidity, builds 150 hand-designed tasks across six cognitive categories with Blender generators randomising speed, lighting and camera angle while preserving each task's structure, and releases a 1.5M-sample training corpus and a 300-question exam. It evaluates 14 video models — 3 reference-to-video, 7 edit, 4 continuation — including its own 16B PWM-WROP, which in a blind pairwise Elo study ranks first among continuation models and third overall (source).

The framing matters more than the rank: it treats a cognitive prior as a trainable target with a held-out exam, where the branch above has no exam at all.

For agents, predicting the tool response turned out to be the wrong target (2026-09-23)

Agent-Editing World Model: Rethinking World Modeling for LLM Agents argues that reconstructing high-entropy, execution-dependent tool responses "offers limited value when real feedback is available", and models how reasoning and actions shape future task progress instead. Reported: 70.5% macro-F1 on its own Action Judge benchmark, 10.6 points over the strongest frontier baseline, and 3.2–6.7 points average improvement across six benchmarks and three agent backbones (source).

Domain world models continue as training substrate

Cosmos-H-Dreams (NVIDIA, surgical robotics) and Qwen-Drive-1.0-4B (Alibaba / Qwen AI Lab, driving) are the form this wiki has held longest: a world model built to train or evaluate an embodied policy, not to be watched. See Embodied Agents.

Open Problems

2026-10-28: Genie 4 adds a vendor-described reasoning loop around interactive audiovisual simulation. Its launch explicitly acknowledges drift and physical/spatial inaccuracies, so plausible rendering remains distinct from verified prediction. (source)

2026-10-11: Embodied Turing Machines — code-only robot policies makes accurate explicit state a prerequisite for a code-only robot policy. This is a different representation choice from generating a plausible future video; its claimed limit is the correctness of state estimation and code. (source)

2026-10-10: Memento 3: Model-Based Recursive Self-Improvement through Reflective Rulebooks verifies revised executable world models against observed transitions. This makes the boundary between replay consistency and unseen-state generalization explicit; passing the former does not establish the latter. (source)

• Drift over an unbounded horizon. "Generates indefinitely" is the claim on GWM Worlds 2; whether the world stays consistent over that horizon is the interesting question and nothing read addresses it.
• No shared benchmark across the two readings. WROP scores video models on a cognitive exam; Agent-Editing scores an agent on task success. There is no instrument on which GWM Worlds 2 and Agent-Editing World Model: Rethinking World Modeling for LLM Agents could be compared, and none is proposed.
• Provenance. A model emitting arbitrary-length photoreal video and audio under live control has no stated watermarking in the captured Runway announcement. Contrast Content Provenance (AI output marking), where SynthID is stated for Gemini 3.8 Live's generated avatars.
• Whether fidelity and controllability trade off, and where. Asserted by Runway, unquantified by anyone.

Key Papers

• 2609.28654 — Training Object Permanence in World Models (WROP), 2026-09-23. Released data, exam, weights and training stack. No wiki page — captured here rather than as a page, per the one-off mention rule (source)

• Agent-Editing World Model: Rethinking World Modeling for LLM Agents — Agent-Editing World Model, 2026-09-23

• 2609.28466 — The Past Frames the Future: Memory for Autoregressive Video Generation, 2026-09-23, 37 upvotes. Read in the same snapshot and not promoted: bears directly on the drift problem above and no abstract figure was extracted for it this run (source)

• HappyWorld-Bench — HappyWorld-Bench, 2026-09-21. The first evaluation instrument on this page that spans the generative and predictive senses; 31 systems across three tracks, 70.14% / 73.33% the only figures (source)

2026-09-28 — the field gets priced, by a buyer with no models in it

AMD agreed to acquire World Labs for $8.2 billion, all-stock, closing expected end of 2026 subject to regulatory approvals; founder Fei-Fei Li to become AMD EVP and chief scientist. World Labs is described across the reporting as building spatial-intelligence models that generate, reconstruct and simulate interactive 3D environments from text, image and video, plus robotic learning and simulation (source).

Two things about this belong on this page and not on the entity pages.

First, it is a valuation of the field, not of a result. Every artefact in ## State of the Art above is either a research preview with no benchmark (GWM Worlds 2, announced and not released) or a benchmark showing current systems failing at object permanence. $8.2B — AMD's second-largest acquisition ever, a sixth of the Xilinx deal — arrives attached to no published score, no named model and no licence, because World Labs published none that anything read names. The field's price and the field's measured capability are being set by different processes.

Second, this page did not know World Labs existed. It has tracked world models since May and the lab is absent from every entry above, from sources.yaml at every tier, and from Embodied Agents. The capture is +2 days, and only because an AI newsletter covers semiconductor deals. Runway was +23 days four days earlier for the same reason. The gap this page has is not in its analysis of world models; it is in which organisations it watches.

Related Concepts

• World Labs — the lab, and the acquisition that put it here
• Embodied Agents — where predictive world models are put to work
• Agents (LLM Agents) — the memory and state-tracking problems the agent branch shares
• Content Provenance (AI output marking) — the unanswered question on the generative branch