Skip to content

Formats

Research latest news

Game AI papers and technical reports worth reading: a selection on agents, environments, generation and player research.

160 picks75 in the last 30 days242 collected in all

Updated

Picks archive · Page 5

Oct 22Thu81–100
  1. arXiv Game AI45

    AnyGameRenderer proposes entity-consistent generative rendering across games and an entity-level benchmark

    AnyGameRenderer is a video-to-video generative renderer that turns prototype footage such as gray-box and colored-geometry scenes produced by AI agents into high-quality visuals with characters, motion detail, and effects, while keeping the layout, camera, and each entity's position and timing unchanged.

    Why it matters: The paper presents a method for turning gray-box prototype videos into high-quality visuals, along with an entity-level benchmark and quantitative comparisons, making it possible to judge whether generative rendering preserves entity correspondences.

  2. arXiv Game AI50

    HiSTEVE 1.0: human-interface VLA agents for open-world games

    HiSTEVE 1.0 is a hierarchical VLA agent for open-world games such as Minecraft, in which a single VLA executes all subtasks using pure visual input and native keyboard-and-mouse actions. The paper reports that it scores 85.0% on MineStudio Bench and achieves an average success rate of 37.4% on OpenHA, with state-of-the-art performance on atomic tasks. The work also reports that the agent can synthesize long-horizon tasks such as an iron pickaxe from scratch, like a human player.

    Why it matters: The paper combines high-level subtask planning and native keyboard-and-mouse visual control in one VLA agent, making it easier to compare the trade-off between long-horizon autonomy and the reliability of interface operation.

  3. arXiv Board & Classic Game AI35

    D-Skill: Distilling teacher knowledge through agent skills

    D-Skill distills the paired trajectory differences between a teacher and a student model into interpretable natural-language skill documents. The student uses these documents at inference time without updating its weights. The paper reports that the method closes 75%–95% of the performance gap between teacher and student on SpreadsheetBench-Verified, ALFWorld, and chess puzzles. Within the same model family, the student's per-task inference cost is 5.7x–8.3x lower than the teacher's.

    Why it matters: Unlike traditional distillation, which rewrites the student's weights, it records the teacher's experience as readable skill documents, making the approach easier to examine and compare.

  4. arXiv Board & Classic Game AI35

    ToM2Vec: Continuous theory-of-mind tokens for multi-agent coordination

    ToM2Vec learns theory of mind for multi-agent coordination through a fixed budget of continuous reasoning tokens, using partners' actual thoughts as training supervision. Across cooperative, competitive, and mixed-motive environments, including Overcooked-AI, multi-party negotiation, Decrypto, Hanabi, and Collab Escape/Capture, it delivers consistent gains over the base model and several competitive baselines.

    Why it matters: The article compresses theory of mind into a fixed number of continuous reasoning tokens, supervised by partners' actual thoughts, in contrast to text-based reasoning approaches.

  5. arXiv Game World Models & Generation39

    LiveAct: heterogeneous action interfaces for real-time multi-agent world models

    LiveAct introduces a heterogeneous action interface that lets a multi-agent video world model generate joint visual futures for multiple controlled agents, handling control inputs ranging from game buttons to robot commands. The method reaches agent attribution accuracy of 0.898 on games and 0.942 on multi-robot manipulation, without relying on agent position or identity cropping. Its causal generator runs at 17.70 FPS at 512×448 resolution.

    Why it matters: The work binds each action stream to its corresponding visual agent and reports streaming generation at 17.70 FPS, making it a useful comparison point for existing single-agent interactive world models.

  6. arXiv Game AI38

    Scaling in games: continued pre-training for embodied agents in diverse virtual worlds

    The paper systematically studies how data volume, diversity, and model scale affect embodied agents in continued pre-training, using 120 training configurations and models from 0.8B to 27B parameters. Diversity improves transfer to related worlds but gives limited gains for worlds that differ substantially. Larger models improve data efficiency, but when trained on a single world they generalize worse to distant worlds. The authors argue that building general-purpose embodied agents requires training stages richer than continued pre-training.

    Why it matters: The paper compares the effects of data volume, diversity, and model scale across 120 training configurations. Its finding that gains from cross-world transfer are limited makes it a useful reference for game agent training strategies.

  7. arXiv Game AI43

    WorldStream: streaming interactive 3D worlds with hierarchical planning and persistent state

    WorldStream introduces an open-ended 3D world model that can expand playable terrain at runtime, respond to open-ended language instructions, and restore visited locations along with their edits. A hierarchical planner outputs macro and micro structural decisions, which deterministic local code then compiles into terrain, roads, settlements, vegetation, and entities. The world record stores plans, seeds, object identities, mutable state, and boundary constraints, so returning chunks can be recompiled from it. Stateful NPCs also update as they act.

    Why it matters: The paper puts terrain generation, NPC state, and records of edits to visited areas into one shared world state. It is useful for comparing approaches to streaming open-world generation.

  8. arXiv Game World Models & Generation40

    Can LLMs serve as event world models? A study from single-actor environments to multi-actor event worlds

    A paper asks whether large language models can predict future observations from event logs and the actions an agent plans to take, using benchmarks built from human Diplomacy games and real Polymarket trades. When only the agent's own actions are given and other actors strongly shape outcomes, model predictions fall well short, and larger models and longer histories help only modestly. When the recorded actions of all actors are provided, fine-tuning and more data improve predictions, but models still overestimate the success rate of planned actions, while imagined experience from world models still improves policy learning.

    Why it matters: The paper argues that when given only the agent's own actions, models struggle to predict outcomes strongly shaped by other actors, a key gap for event world modeling with multiple actors

  9. arXiv Board & Classic Game AI39

    ToM2Vec: Continuous theory-of-mind tokens for multi-agent coordination

    ToM2Vec learns theory of mind for multi-agent coordination through a fixed budget of continuous reasoning tokens, using partners' actual thoughts as training supervision. Across cooperative, competitive, and mixed-motive environments, including Overcooked-AI, multi-party negotiation, Decrypto, Hanabi, and Collab Escape/Capture, it delivers consistent gains over the base model and several competitive baselines.

    Why it matters: The article compresses theory of mind into a fixed number of continuous reasoning tokens, supervised by partners' actual thoughts, in contrast to text-based reasoning approaches.

Oct 21Wed
  1. arXiv Game World Models & Generation50

    LiveAct: heterogeneous action interfaces for real-time multi-agent world models

    LiveAct introduces a heterogeneous action interface that lets a multi-agent video world model generate joint visual futures for multiple controlled agents, handling control inputs ranging from game buttons to robot commands. The method reaches agent attribution accuracy of 0.898 on games and 0.942 on multi-robot manipulation, without relying on agent position or identity cropping. Its causal generator runs at 17.70 FPS at 512×448 resolution.

    Why it matters: The work binds each action stream to its corresponding visual agent and reports streaming generation at 17.70 FPS, making it a useful comparison point for existing single-agent interactive world models.

  2. arXiv Game AI34

    Neural controlled directional frame generation for sprite animation

    The study proposes a controllable directional frame generation method for sprite animation. It uses three expert models with shared U-Net architectures to handle forward extrapolation, intermediate interpolation, and backward extrapolation. The models were trained on only 280 curated sprite triplets and evaluated on 42 test samples using the FLIP metric, scoring 0.26 to 0.27 across the three directions, 17% to 57% higher than a general-purpose baseline. The method uses deterministic single-pass generation and a more compact model, aimed at production pipelines for pixel-style sprite animation.

    Why it matters: For pixel sprite animation needs such as forward extrapolation, backward extrapolation, and interpolation, the study trains on just 280 curated samples. It offers a useful reference for training on small datasets in niche visual domains.

  3. arXiv Game AI42

    SolarRenderer: Few-Step Generative Reference Rendering Based on Structural Conditions

    SolarRenderer is a structure-conditioned video renderer that turns layouts, actions, and camera trajectories from a game engine or coding agent into realistic video. Built on a frozen MiniMax-H3 33B backbone with two lightweight LoRAs, it renders in four steps, 12.3 times faster than 50-step sampling, and its streaming variant renders 768p video in real time. The paper reports state-of-the-art results on an in-distribution test set and a generalization benchmark covering 16 structural representations, and generalization to 11 unseen representations such as pose skeletons, sketches, and point clouds.

    Why it matters: The paper turns the coarse structure output by a game engine into video, and its four-step sampling is 12.3 times faster than 50 steps, making it a reference for real-time generative rendering.

  4. arXiv Game World Models & Generation41

    ActionRoPE: Using world coordinates for spatially consistent control in planar-camera game world models

    ActionRoPE turns action-driven camera displacement into world-coordinate rotary positions, so tokens at the same world position keep the same positional encoding over time. The paper also introduces Coordinate-Conditioned Noise Scheduling (CNS), which stabilizes observed regions earlier and generates new regions by referencing them. Compared with four representative action-conditioning mechanisms, the method follows commanded actions more faithfully and cuts revisit drift by 66% relative to the strongest baseline. It also transfers to video backbones with 1.3B to 33B parameters and to out-of-distribution scenarios.

    Why it matters: Converting action displacements into world-coordinate rotary positions is worth a look for action control in world models of top-down and isometric games.

  5. arXiv Game AI46

    Fig: automated multiagent behavior elicitor

    Fig is an agent system that intervenes in game environments in a targeted way to elicit six types of target behavior: cooperation, exploitation, deception, tacit coordination, betrayal, and sabotage. The paper reports that across valid scenarios generated by Fig, models attempted the target behavior in an average of 45.2% of rounds. For game-and-behavior combinations with a control, interventions raised the attempt rate by an average of 10.9% over matched control games. The authors argue that multiagent environments can serve as a programmable, controllable, and reproducible testbed for behavior evaluation, and the code is publicly available on GitHub.

    Why it matters: It turns evaluation of multi-agent social behavior from passive observation into programmable environment intervention, useful for researchers who want to design reproducible test environments.

  6. arXiv Game AI42

    WorldStream: streaming interactive 3D worlds with hierarchical planning and persistent state

    WorldStream introduces an open-ended 3D world model that can expand playable terrain at runtime, respond to open-ended language instructions, and restore visited locations along with their edits. A hierarchical planner outputs macro and micro structural decisions, which deterministic local code then compiles into terrain, roads, settlements, vegetation, and entities. The world record stores plans, seeds, object identities, mutable state, and boundary constraints, so returning chunks can be recompiled from it. Stateful NPCs also update as they act.

    Why it matters: The paper puts terrain generation, NPC state, and records of edits to visited areas into one shared world state. It is useful for comparing approaches to streaming open-world generation.

  7. arXiv Game AI51

    SolarRenderer: Few-Step Generative Reference Rendering Based on Structural Conditions

    SolarRenderer is a structure-conditioned video renderer that turns layouts, actions, and camera trajectories from a game engine or coding agent into realistic video. Built on a frozen MiniMax-H3 33B backbone with two lightweight LoRAs, it renders in four steps, 12.3 times faster than 50-step sampling, and its streaming variant renders 768p video in real time. The paper reports state-of-the-art results on an in-distribution test set and a generalization benchmark covering 16 structural representations, and generalization to 11 unseen representations such as pose skeletons, sketches, and point clouds.

    Why it matters: The paper turns the coarse structure output by a game engine into video, and its four-step sampling is 12.3 times faster than 50 steps, making it a reference for real-time generative rendering.

  8. arXiv Game AI41

    OverSeer: Multi-agent world models beyond what agents see

    OverSeer is a multi-agent world model that adds fixed-position auxiliary camera views beyond each agent's first-person view, improving temporal and cross-agent consistency in multi-agent video generation. It uses a delta-VAE that encodes only changes relative to a near-static reference image, reducing the number of tokens needed to process the auxiliary views. Experiments in Minecraft and Doom show improved world and agent consistency, and qualitative results on real-world multi-camera capture indicate the method does not depend on a game engine.

    Why it matters: Multi-agent video world models struggle to stay consistent across viewpoints; this work uses a fixed auxiliary viewpoint as a proxy for world state, which is a useful reference for multi-view generation.

Oct 20Tue
  1. arXiv Game AI58

    Scaling in games: continued pre-training for embodied agents in diverse virtual worlds

    The paper systematically studies how data volume, diversity, and model scale affect embodied agents in continued pre-training, using 120 training configurations and models from 0.8B to 27B parameters. Diversity improves transfer to related worlds but gives limited gains for worlds that differ substantially. Larger models improve data efficiency, but when trained on a single world they generalize worse to distant worlds. The authors argue that building general-purpose embodied agents requires training stages richer than continued pre-training.

    Why it matters: The paper compares the effects of data volume, diversity, and model scale across 120 training configurations. Its finding that gains from cross-world transfer are limited makes it a useful reference for game agent training strategies.

  2. arXiv Game AI69

    Matrix-Game 4.0 harnesses video models to build real-time playable game worlds

    Matrix-Game 4.0 separates game execution from visual generation. A lightweight game engine maintains the world state and executes interaction rules, while an autoregressive video model turns low-polygon rendered frames into detailed images. The paper reports real-time 480p gameplay at 60 FPS on a single NVIDIA RTX5090, with a key-press-to-display latency of 112 ms. Single-step generation sustains 83 FPS throughput, with peak GPU memory usage of 18.2 GiB.

    Why it matters: This work hands world state and interaction rules to an executable engine and leaves image generation to a video model, offering a useful reference for the engineering split in playable world models.

  3. arXiv Game AI35

    World2HOSI uses a multi-expert harness to generate long-horizon human-object-scene interaction data

    World2HOSI is a multi-expert harness framework that generates continuous human-object-scene interaction (HOSI) data from 3DGS scenes and natural language instructions. The paper reports that it achieves the best task completion and interaction quality on the evaluated HOSI tasks, and produces more diverse data. The method reduces physical artifacts through persistent scene state, bounded residual optimization, and contact-driven physics simulation.

    Why it matters: The paper orchestrates existing expert models within a unified scene state, offering a transferable idea for generating long-horizon interactions where training data is scarce.