From game generation to game playing: Script2Play trains VLA models with fully automated synthetic demonstrations
From Game Generation to Game Playing: Fully Automated Ability Training for Vision-Language-Action Models
The authors propose a framework that automatically generates game environments and expert demonstrations, and use it to train a game-playing policy, Script2Play. In this framework, an LLM agent calls game-building tools to construct executable environments and tasks, and a procedural expert then draws on the simulator's privileged state to produce aligned instructions, rendered observations, and keyboard and mouse actions.
The framework strings environment generation, expert demonstration and behavior cloning into a single pipeline, so readers can judge whether synthetic supervision can stand in for human demonstrations.
Source: arXiv Game AI · arxiv.org