Scaling in games: continued pre-training for embodied agents in diverse virtual worlds
Scaling in Games: Continued Pre-Training For Embodied Agents In Diverse Virtual Worlds
The paper systematically studies how data volume, diversity, and model scale affect embodied agents in continued pre-training, using 120 training configurations and models from 0.8B to 27B parameters. Diversity improves transfer to related worlds but gives limited gains for worlds that differ substantially. Larger models improve data efficiency, but when trained on a single world they generalize worse to distant worlds. The authors argue that building general-purpose embodied agents requires training stages richer than continued pre-training.
The paper compares the effects of data volume, diversity, and model scale across 120 training configurations. Its finding that gains from cross-world transfer are limited makes it a useful reference for game agent training strategies.
Source: arXiv Game AI · arxiv.org