Skip to content

Fields

Game agents latest news

AI that acts in games: generalist game agents, LLM-driven bots, agents in Minecraft, StarCraft and other environments, benchmarks and open frameworks.

111 picks55 in the last 30 days170 collected in all

Updated

Picks archive · Page 2

Nov 9Mon21–40
  1. arXiv Game AI58

    ViCoR introduces visual conditioned reflexes to cut decision latency for real-time VLM game agents

    The study proposes ViCoR, in which a main VLM handles routine decisions while a fast visual reflex branch handles sudden reactions; on the GameWorld benchmark's 17 2D/3D games, the approach cuts average per-step decision latency from 14.0 s to 5.4 s.

    Why it matters: The paper compares latency and success rates across 17 games, offering a performance reference for real-time VLM agents.

Nov 7Sat
  1. arXiv Game AI44

    AVCORP builds a 5,000-utterance annotated Avalon corpus to train models to detect deception

    The research team built the AVCORP corpus, comprising 5,000 annotated utterances from 250 Avalon games, labeled with 37 strategies mapped to a 4x4 behavior matrix, along with recorded abductive hypotheses, suspicion levels, and nested theory-of-mind beliefs.

    Why it matters: The paper releases 5,000 annotated Avalon utterances with a taxonomy of 37 strategies, plus detection comparisons against frontier models on the same test set.

Nov 6Fri
  1. arXiv Board & Classic Game AI42

    LLM hand-sets network weights zero-shot, building 10 networks across a two-player game difficulty ladder

    The paper has a large language model hand-set all the weights of feedforward networks without using gradient descent or training data at any point, across a four-level difficulty ladder spanning Nim, tic-tac-toe, two sizes of Connect Four and a chess endgame, building 10 networks in total and independently verifying each one.

    Why it matters: The paper reports the construction and independent verification results for 10 hand-weighted networks, and the method's failure on multi-move checkmates in the chess endgame can serve as a reference point for judging its limits.

  2. arXiv Game AI71

    PlayGen2: Learning real-time playable worlds with explicit 3D state

    The authors propose PlayGen2, a real-time interactive world model that separates learned 3D state evolution from neural rendering, using a volumetric occupancy grid and an identity-indexed 3D state to maintain persistent world memory. Tested on Doom, the model learned world dynamics including geometry-aware player movement and player-NPC interaction, and the full simulation and rendering pipeline runs at about 160ms backend latency on a MacBook Pro with an M4 Pro chip.

    Why it matters: The paper splits a playable world into explicit 3D state evolution and neural rendering, and reports measured latency and zero-shot transfer results, making it easier to judge whether such world models can support real-time interaction.

  3. Grok Bot Research42

    CrowdDirector: Commanding crowds of Unity NPCs in real time with natural language, invoking the LLM only when instructions change

    CrowdDirector is a framework for directing virtual crowds with natural language at runtime in a simulation. It calls the LLM only when the user adds or modifies an instruction, and the LLM turns the request into structured instructions, which a scene validator then checks to confirm that the entities and actions referenced exist in the current Unity scene.

    Why it matters: The paper presents a crowd NPC control scheme that invokes the LLM only when instructions change, offering a middle ground between fixed rules and always-on LLM control.

Nov 5Thu
  1. arXiv Board & Classic Game AI38

    arXiv paper finds frontier LLM agents still struggle with zero-shot coordination in minimal environments

    The paper studies zero-shot coordination (ZSC) among LLM agents and finds that in deliberately simplified minimal environments, frontier LLM agents still struggle to coordinate. The main causes are their limited understanding of the coordination problem and weak reasoning about partners' beliefs. The authors propose coordination-friendly definitions (CFDs) as a way to enable stable coordination among LLM agents, and show that these definitions can be discovered automatically without human design; the code is released on GitHub.

    Why it matters: The paper pinpoints the reasoning gaps behind frontier LLM agents' zero-shot coordination failures in minimal environments, and proposes coordination-friendly definitions that can be discovered automatically.

  2. Youxituoluo44

    When AI agents pick games for players, independent developers become even harder to find

    AI agents such as DSH (DeepSeek Harness) are being used to actively search for and recommend games on players' behalf, but Youxi Tuoluo argues this is not necessarily good news for indie developers. AI recommendations rely on public information such as store pages, media reviews and player comments, so publishers that already have a marketing apparatus keep their advantage, while small teams with only a Steam store page and a trailer are hard for AI to discover.

    Why it matters: This piece walks through cases from DSH scanning Bilibili to Meta Muse and Google Play Game Finder, examining why AI recommendations may not lower the exposure cost for indie developers.

Nov 4Wed
  1. arXiv Game AI43

    AceAction-5M: A dataset for hierarchical-goal generalist action learning across multiple 3D game worlds

    The authors present the AceAction dataset, a collection of human gameplay demonstrations across multiple 3D games that links overall task instructions, ordered subgoals and local skill instructions with time-aligned visual observations and keyboard-and-mouse actions, covering navigation, reconnaissance, combat, interaction and recovery. The team uses a vision-language model pipeline to identify subgoals from changes in goals or interactable objects, anchor navigation goals to recognizable waypoints, and compose adjacent subgoals into task instructions. Review of 500 trajectories achieved a 95.8% pass rate in independent evaluation.

    Why it matters: The dataset strings overall task instructions, ordered subgoals and local skill instructions together with time-aligned frames and keyboard-and-mouse actions, so it can be used to observe how an agent executes, composes and switches skills.

Nov 3Tue
  1. arXiv Board & Classic Game AI46

    How Maia-2 simulates human play across skill levels

    The paper studies how the chess Transformer Maia-2 simulates players of different levels through skill conditioning, and, holding the chess position and model weights fixed, analyzes the positions where its prediction shifts from a blunder to the best move. The researchers trained probes to predict from Maia-2's internal states whether 172 chess concepts are present; probe accuracy was nearly unchanged across skill conditions, and a probe trained at one level transferred accurately to other levels.

    Why it matters: The paper uses probes and last-layer fine-tuning to disentangle the effect of skill conditioning on Maia-2, offering a reference for player modeling that separates knowledge retention from behavioral expression.

Nov 1Sun
  1. arXiv Board & Classic Game AI38

    Paper explores whether LLM agents can learn by watching tutorial videos, proposes TAL paradigm

    The paper proposes TAL (Tutorial Agentic Learning), a parameter-free continual learning paradigm that organizes tutorial videos into persistent structured tutorial memory, enabling LLM agents to learn from tutorial videos without updating their parameters.

    Why it matters: The paper replaces vector similarity with directory-tree retrieval to turn tutorial videos into reusable memory, and this parameter-free continual learning approach can be transferred to other domains.

  2. arXiv Board & Classic Game AI43

    Paper explores whether LLM agents can learn by watching tutorial videos, proposes TAL paradigm

    The paper proposes TAL (Tutorial Agentic Learning), a parameter-free continual learning paradigm that organizes tutorial videos into persistent structured tutorial memory, enabling LLM agents to learn from tutorial videos without updating their parameters.

    Why it matters: The paper replaces vector similarity with directory-tree retrieval to turn tutorial videos into reusable memory, and this parameter-free continual learning approach can be transferred to other domains.

Oct 30Fri
  1. GamesBeat71

    Ramen launches open-source autonomous game testing agent Aura Autopilot with support for Unreal Engine, Unity and Godot

    Ramen released Aura Autopilot, an open-source game playtesting agent that can play games in real time in Unreal Engine, Unity and Godot and learn during play, along with a whitepaper disclosing its architecture and benchmarks. The company says it runs 3 times faster than Aura's Verification Agent at a lower cost, can be used with Claude Code and Codex subscriptions or a bring-your-own API key, and the project is now live on GitHub.

    Why it matters: The original article details how Aura Autopilot connects open-source to Unreal Engine, Unity and Godot, and its speed comparison, which can help readers judge whether it fits their own QA workflow.

  2. arXiv Game World Models & Generation58

    Wizard: Manipulate any world in real time via agentic control

    The paper proposes Wizard, an agentic game world model that can precisely control character actions in real time and achieve zero-shot generalization to new scenes, characters, and actions. The authors built a dataset covering 750 scenes and 178 characters, using skeleton and geometry information as structured representations to condition a video generation model, and introduced an adaptive masking mechanism to improve the consistency between the conditioning signal and the generated video. A companion agent framework explicitly maintains and updates game state, letting players freely configure virtual worlds and control character actions at fine granularity.

    Why it matters: The paper presents a dataset of 750 scenes and 178 characters and a structured agentic conditioning scheme, offering a look at an implementation path for real-time controllable world models.

  3. arXiv Game World Models & Generation58

    Algebraic World Model: Turning game environment interactions into Lean 4-provable mathematical models

    Researchers propose the Algebraic World Model (AWM), which turns interactions with unfamiliar environments into executable Python hypotheses, then organizes them into typed objects and shared rules, simplifies the structure with category-theory-guided transformations, and finally formalizes it in Lean 4 to prove reachability and impossibility.

    Why it matters: The paper presents a pipeline that turns interactions with unfamiliar game environments into Lean 4-checkable mathematical models, letting readers understand how formal proofs can be used to determine level unreachability.

  4. arXiv Game AI58

    Fight World Model: Customizable combat through visual interaction proxies

    The authors propose Fight World Model (FWM), which separates rule-constrained combat execution from video rendering: an executable engine maintains state and resolves interactions, and a deterministic compiler then encodes the results into spatiotemporally aligned interaction proxies.

    Why it matters: The paper separates rule execution from video rendering, letting readers understand how a programmable combat world model keeps hit and health states consistent.

  5. arXiv Game World Models & Generation60

    Algebraic world model AWM proposes a framework from environment exploration to Lean 4 formal proofs

    The research team proposes the Algebraic World Model (AWM), which turns interaction with an unknown environment into a mathematical model that can be formally reasoned about in Lean 4. On two unsolvable game levels, six AWM runs all produced kernel-verified, closed Lean proofs ruling out every valid winning sequence, whereas Claude Opus 5.5 and GPT-6 Astra, equipped with programming tools, failed to produce a proof in twelve baseline runs.

    Why it matters: Reaching a kernel-verified unreachability conclusion on levels that cannot be completed shows how formal reasoning can be applied to modeling game environments.

  6. arXiv Game World Models & Generation58

    AWM: an algebraic world model that turns unfamiliar game environments into Lean-provable mathematical models

    Researchers propose the Algebraic World Model (AWM), which turns interactions with unfamiliar environments into explicit mathematical models that support formal reasoning, and proves reachability and unreachability in Lean 4.

    Why it matters: The paper presents a pipeline for turning unfamiliar game environments into checkable mathematical models and uses Lean to prove levels unreachable, helping readers understand how formal reasoning can enter game agents.

  7. arXiv Game World Models & Generation58

    Wizard: Manipulate any world in real time via agentic control

    The paper proposes Wizard, an agentic game world model that can precisely control character actions in real time and achieves zero-shot generalization to new scenes, characters, and actions. The researchers built a dataset covering 805 scenes and 178 characters, using skeleton and geometry information as structured proxies to condition video generation, and supporting character-aware adaptive masking and world-anchored spatial conditions through explicit tracking and updating of game state.

    Why it matters: The paper provides a dataset of 805 scenes and 178 characters along with the WBench evaluation, offering a concrete reference for comparing existing world models on real-time controllable generation.

Oct 29Thu
  1. arXiv Game AI40

    TaskFrame proposes hierarchical planner and controller framework connecting task-level planning to frame-level keyboard-and-mouse control

    Guangzhi Wang proposes TaskFrame, a hierarchical planner and controller framework that connects task-level planning with frame-level keyboard and mouse control. A vision-language task planner breaks high-level goals into ordered atomic instructions, optimized with supervised learning and dependency-aware GRPO; the hidden state of a special goal token that terminates each atomic instruction is cached as a continuous goal embedding and reused in subsequent control steps.

    Why it matters: The paper presents a three-stage training pipeline for the planner and controller along with a cached goal embedding approach, so readers can judge how the hierarchical structure connects vision-language reasoning with keyboard-and-mouse control.

  2. arXiv Board & Classic Game AI36

    RetroChessBench: A benchmark for reconstructing and proving chess game histories

    The authors propose RetroChessBench, a retrospective chess benchmark with 12 tasks covering fixed-length construction, unique final positions and exhaustive answer sets, together with reference answers, grading rubrics and a terminal-reward environment.

    Why it matters: The paper presents 12 retrospective chess tasks with grading rubrics and documents the specific failure types of frontier models on pure rule-based reasoning, serving as a reference for evaluating reasoning ability.