Skip to content

Fields

World models latest news

Interactive generative environments and world models: playable worlds generated in real time, physics and consistency, and what they mean for game engines.

92 picks47 in the last 30 days123 collected in all

Updated

Picks archive · Page 4

Oct 20Tue61–80
  1. arXiv Game AI69

    Matrix-Game 4.0 harnesses video models to build real-time playable game worlds

    Matrix-Game 4.0 separates game execution from visual generation. A lightweight game engine maintains the world state and executes interaction rules, while an autoregressive video model turns low-polygon rendered frames into detailed images. The paper reports real-time 480p gameplay at 60 FPS on a single NVIDIA RTX5090, with a key-press-to-display latency of 112 ms. Single-step generation sustains 83 FPS throughput, with peak GPU memory usage of 18.2 GiB.

    Why it matters: This work hands world state and interaction rules to an executable engine and leaves image generation to a video model, offering a useful reference for the engineering split in playable world models.

  2. arXiv Game AI44

    InfiniDream: Open-ended photorealistic 3D worlds through procedural generation

    InfiniDream proposes a VLM-driven framework for open-ended procedural 3D world generation that creates novel photorealistic assets without relying on preset factories and APIs. Evaluated on 100 object and 20 scene generation tasks, it shows improvements on four perceptual metrics. The framework also combines semantic scene planning with deterministic frustum-guided grid placement to build large-scale 3D worlds with procedural population and animation.

    Why it matters: Existing VLM-guided procedural methods are mostly limited by preset factories and APIs; this paper tries to open up asset types and can be read alongside comparable methods.

Oct 19Mon
  1. Viggle AI35

    Viggle AI releases JST-2, a physics-native world foundation model

    Viggle AI has released JST-2, a full-stack physics-native world foundation model. According to the announcement, its entire model stack is built in 3D rather than in pixel or video space. The company says it can generate 3D scenes within seconds, scale them in real time, and create characters with built-in motion using the same general architecture.

    Why it matters: Unlike most world models that learn in pixel or video space, it builds its entire model stack directly in 3D, making it a useful reference for comparing different technical approaches.

  2. arXiv Game World Models & Generation49

    ActionRoPE: Using world coordinates for spatially consistent control in planar-camera game world models

    ActionRoPE turns action-driven camera displacement into world-coordinate rotary positions, so tokens at the same world position keep the same positional encoding over time. The paper also introduces Coordinate-Conditioned Noise Scheduling (CNS), which stabilizes observed regions earlier and generates new regions by referencing them. Compared with four representative action-conditioning mechanisms, the method follows commanded actions more faithfully and cuts revisit drift by 66% relative to the strongest baseline. It also transfers to video backbones with 1.3B to 33B parameters and to out-of-distribution scenarios.

    Why it matters: Converting action displacements into world-coordinate rotary positions is worth a look for action control in world models of top-down and isometric games.

  3. QbitAI57

    VAST open-sources EdenLive real-time world model, over 24 FPS at 720P on a single consumer GPU

    VAST has open-sourced EdenLive, a real-time world model that generates frames live under any reference input and art style and keeps running continuously. According to VAST, on a single Nvidia RTX 5090 at 720P, its 5B model exceeds 24 FPS, with end-to-end latency in the hundreds of milliseconds. EdenLive handles only visual rendering, while world state and interaction rules are still maintained by the upstream interaction system. Model weights, inference code and a technical report are now available.

    Why it matters: The article explains the design split between real-time rendering and state maintenance, and gives speed and latency figures on consumer GPUs, helping readers judge its usable range in game prototyping.

Oct 17Sat
  1. arXiv Game AI43

    Mirror: self-improving video-to-action models

    Mirror is a self-supervised method that learns video-to-action mapping through interaction with the environment, without prior action data or hand-written rewards. On Atari, OGBench, and DeepMind Control, it outperforms mainstream observation-based learning methods using its learned video plans. On Minecraft BASALT, it surpasses methods that have access to expert actions or hand-crafted task knowledge. In Minecraft survival tasks, it obtained an iron pickaxe through self-improvement alone, with no reward or expert action labels.

    Why it matters: It doesn't rely on human action annotations or hand-written rewards, letting video planning models become executable agent policies. Worth a look for readers interested in combining video models with agents.

  2. arXiv Game AI45

    Mirror: self-improving video-to-action models

    Mirror is a self-supervised method that learns video-to-action mapping through interaction with the environment, without prior action data or hand-written rewards. On Atari, OGBench, and DeepMind Control, it outperforms mainstream observation-based learning methods using its learned video plans. On Minecraft BASALT, it surpasses methods that have access to expert actions or hand-crafted task knowledge. In Minecraft survival tasks, it obtained an iron pickaxe through self-improvement alone, with no reward or expert action labels.

    Why it matters: It doesn't rely on human action annotations or hand-written rewards, letting video planning models become executable agent policies. Worth a look for readers interested in combining video models with agents.

Oct 15Thu
  1. arXiv Game AI39

    PAWA: Panoramic action-conditioned world modeling for multiple agents

    PAWA presents a synchronized POV and panoramic video dataset and framework for action-conditioned multi-agent world modeling in multiplayer Minecraft scenarios. The model uses a two-tower structure to jointly predict two video streams, and cross-attention aligns tokens facing the same direction in a shared world coordinate system. The paper reports that its POV prediction quality outperforms existing multiplayer world models while also generating high-fidelity panoramic predictions. The dataset and collection pipeline are open-sourced.

    Why it matters: The paper fills a gap with synchronized POV and panoramic video data for multiplayer settings, making it a useful reference for readers focused on building datasets for multiplayer world models.

  2. arXiv Game AI35

    SMART: Long-horizon video generation with slot memory addressed by region tokens

    SMART proposes a slot memory addressed by region tokens, supporting spatial recall in long-horizon video generation at fixed storage and compute cost. Writes are addressed by the observation camera and reads are queried by the rendering camera, with no explicit 3D representation or growing cache needed. Each read also yields a confidence score that modulates how much the chunk-causal diffusion Transformer trusts retrieved content. On long-trajectory tests in Memory Maze and Minecraft, it maintains high-fidelity recall beyond the training horizon.

    Why it matters: Compared with memory approaches that rely on explicit 3D reconstruction or temporal recurrence, this method uses fixed-size slot memory addressed by camera position, which is worth a look for readers following long-horizon world models.

Oct 14Wed
  1. arXiv Game World Models & Generation35

    Paper proposes CGWMs, learning interpretable world model concept representations with temporal logic

    The paper proposes Concept-Grounded World Models (CGWMs), which learn interpretable concept representations from observations using complete knowledge expressed in temporal logic rather than black-box representations. Planning is formulated as differentiable inference, optimizing agent actions via gradient ascent to satisfy goals. In comparisons with pure-neural and neurosymbolic baselines in two game environments, CGWMs' concept grounding is more accurate, holds over long horizons as well, and plans more robustly both in and out of distribution.

    Why it matters: The paper places temporal-logic knowledge in the same framework as concept learning and planning for world models, and compares against pure-neural and neurosymbolic baselines in two game environments, making it a basis for judging the method trade-offs of this line of work.

Oct 13Tue
  1. QbitAI38

    ByteDance world model debuts live at 2026 Wonderful Night, generating and freely exploring 3D scenes from a single sentence

    At the tech segment of Douyin Group's 2026 Wonderful Night, ByteDance showed its self-developed world model, which can generate a 3D scene from a single sentence and supports free exploration, real-time interaction, object generation and editing, and cross-scene connectivity. The model has not been officially released yet, and the live demo was presented by Tim of Yingshi Jufeng.

    Why it matters: Using ByteDance's world model live demo as an entry point, this compares the differing approaches of OpenAI, DeepMind and Nvidia to spatial modeling and physical AI training.

Oct 12Mon
  1. QbitAI61

    Huawei's Memento 3 clears all 25 games in ARC-AGI-3 with 56% fewer actions than humans

    Huawei's Noah's Ark Lab, with UCL and other teams, proposed Memento 3, which cleared all 25 games and 183 levels in the public ARC-AGI-3 set, scoring a perfect 100 RHAE. It does not update model weights or fine-tune for specific games; using Claude Opus 5 as its base model, it recursively self-improves through a continuously updated Markdown rulebook and an executable Python world model, completing the run in 7,518 total actions — about 56% fewer than the 17,135 taken by humans.

    Why it matters: The method does not update model weights, relying instead on an external rulebook and an executable world model for continuous iteration, offering a transferable approach to optimizing agent efficiency under unknown rules.

  2. arXiv Game AI43

    Esports World Model dataset (EWM)

    EWM is a dataset of human matches across StarCraft II, Dota 2, and Counter-Strike 2, comprising 3,053 matches and maps and about 4,880 hours of player-perspective footage. The duration of the CS2 portion is an estimate.

    Why it matters: The dataset links human decisions, shared state, and each player's private field of view. Researchers studying controllable multiplayer world models can draw directly on its data organization.

Oct 11Sun
  1. The Decoder65

    Odyssey opens world model Odyssey-3 in public research preview with a free online demo

    Odyssey released a public research preview of its world model Odyssey-3, letting users generate interactive environments from text prompts and explore them in real time, with developers able to apply for API access. The free online demo runs on Odyssey-3 Flash; the base model has 1.4 billion parameters and generates 832 × 480 video, while Odyssey-3 Pro supports 1280 × 720.

    Why it matters: On physics benchmarks, the original article gives both the official self-selected single-run score and the four-round average, so readers can judge the differences in statistical reporting for such world model results.

Oct 9Fri
  1. arXiv Game AI56

    MultiWorldBench: can independently controlled views describe one shared world

    The authors propose MultiWorldBench, a diagnostic Minecraft benchmark for testing whether independently controlled views in a multi-player world model stay consistent with one single persistent shared world; it contains 495 case configurations, seven task suites and ten capabilities.

    Why it matters: With 495 configurations, the paper compares three generative world models against the reference Engine GT on ten capabilities, and the score gaps show where the current weaknesses lie.

  2. arXiv Game AI45

    AgentGarten: a code world framework for continually evolving agents

    The paper proposes the AgentGarten framework, which couples simulators and game engines to a shared neural renderer to build real-time interactive virtual environments. The simulation backend maintains persistent world state and executes program-defined interaction rules, while the renderer generates visual observations from structured conditions exported through a unified interface; the neural renderer is adapted from a pretrained video model to take geometry-conditioned input, is distilled with the proposed Adversarial Forcing, and has its inference optimized for real-time interaction.

    Why it matters: The paper wires simulators and game engines into a single neural renderer, and reports learning-efficiency results comparing an agent's 4 rounds of experience with millions of rounds of reinforcement learning.

  3. arXiv Game AI38

    AgentGarten: Code worlds for evolving agents

    AgentGarten combines simulators, game engines, and a shared neural renderer to build real-time interactive environments, in which agents perceive the world through visual observations and interact with it. The paper proposes Adversarial Forcing, which distills a pretrained video model into a geometry-conditioned neural renderer and lets the encoding of historical observations also be updated by losses from subsequent predictions. The experiments report that agents learn from only 4 rounds of experience, whereas the traditional reinforcement learning baseline requires training on the order of millions of steps.

    Why it matters: The paper writes worlds as code and hands them to a shared neural renderer to generate the visuals. Comparing its learning efficiency with traditional reinforcement learning can help you judge whether this approach to expanding environments suits your project.

  4. Odyssey35

    Odyssey releases world model Odyssey-3

    Odyssey has officially announced the release of its world model Odyssey-3, calling it a big step toward having world models power physical AI and generate environments for training intelligences. The company says Odyssey-3 is now available to try and provides the link odyssey.systems/meet-odyssey-3. The tweet did not disclose model size, capability details, or availability scope.

  5. Odyssey35

    Odyssey releases foundation world model Odyssey-3

    Odyssey released the foundation world model Odyssey-3, saying it can support a range of applications including physical AI, human experiences and agent training. The company specifically notes that agents can learn from experience and accomplish goals inside Odyssey-3.

    Why it matters: Odyssey lays out how it positions Odyssey-3, so readers can see how foundation world models are being applied to agent experience learning.