Proposes a VLA game policy framework that auto-generates environments and expert demonstrations
What happened
Changran Liu and co-authors propose a framework that automatically generates game environments and expert demonstrations, and use it to train a game policy model, Script2Play, without human gameplay demonstrations and without reinforcement learning. In this framework, an LLM agent calls game-building tools to construct executable environments and tasks, and a procedural expert then uses the simulator's privileged state to generate instructions and actions aligned with rendered observations and keyboard-and-mouse input as training data. The authors report that Script2Play was obtained by behavior cloning a 4B vision-language model on 150.3 hours of synthetic gameplay data in Minecraft and Roblox.
AI-written from the coverage · updated 2 d ago
Coverage timeline
Follow the coverage to see the story from different sides.
- arXiv Game AIPickFrom game generation to game playing: Script2Play trains VLA models with fully automated synthetic demonstrations
The authors propose a framework that automatically generates game environments and expert demonstrations, and use it to train a game-playing policy, Script2Play. In this framework, an LLM agent calls game-building tools to construct executable environments and tasks, and a procedural expert then draws on the simulator's privileged state to produce aligned instructions, rendered observations, and keyboard and mouse actions.