Scaling study of continual pretraining for embodied agents in virtual worlds
What happened
In an arXiv paper, Kuan Zhang and colleagues systematically studied continual pretraining (CPT) for embodied agents in virtual worlds, using 120 training configurations. The experiments varied three dimensions: data volume, data diversity, and model size (0.8B to 27B). The paper reports that data diversity improves transfer to related worlds but offers limited gains for worlds that differ substantially. Larger models are more data-efficient, but when trained on a single world, they generalize worse to distant worlds. Based on these findings, the authors argue that continual pretraining alone is not enough to build general-purpose embodied agents. Richer training stages beyond it are needed.
AI-written from the coverage · updated 2026-10-22
Coverage timeline
Follow the coverage to see the story from different sides.
- arXiv Game AIPickScaling in games: continued pre-training for embodied agents in diverse virtual worlds
The paper systematically studies how data volume, diversity, and model scale affect embodied agents in continued pre-training, using 120 training configurations and models from 0.8B to 27B parameters. Diversity improves transfer to related worlds but gives limited gains for worlds that differ substantially. Larger models improve data efficiency, but when trained on a single world they generalize worse to distant worlds. The authors argue that building general-purpose embodied agents requires training stages richer than continued pre-training.