Authors propose AceAction-5M, a cross-3D-game action learning dataset
What happened
The authors propose and build AceAction (referred to as AceAction-5M in the newly reported title): a vision-language-action dataset for skill composition and subgoal switching, collected from human play across multiple 3D games. It links overall task instructions, ordered subgoals, and local skill instructions with time-aligned visual observations and keyboard-and-mouse actions, covering navigation, scouting, combat, interaction, and recovery. Annotation is done automatically by a vision-language model pipeline: subgoals are identified from changes in goals or interactable objects, navigation targets are anchored to recognizable waypoints, and adjacent subgoals are composed into task instructions. The authors report that a review of 500 trajectories passed independent review at a 95.8% rate; this is the authors' self-reported annotation-quality result, not a validation of downstream task performance.
AI-written from the coverage · updated 18 d ago
Coverage timeline
Follow the coverage to see the story from different sides.
- arXiv Game AIPickAceAction-5M: A dataset for hierarchical-goal generalist action learning across multiple 3D game worlds
The authors present the AceAction dataset, a collection of human gameplay demonstrations across multiple 3D games that links overall task instructions, ordered subgoals and local skill instructions with time-aligned visual observations and keyboard-and-mouse actions, covering navigation, reconnaissance, combat, interaction and recovery. The team uses a vision-language model pipeline to identify subgoals from changes in goals or interactable objects, anchor navigation goals to recognizable waypoints, and compose adjacent subgoals into task instructions. Review of 500 trajectories achieved a 95.8% pass rate in independent evaluation.
- arXiv Game AIPickAceAction-5M: A dataset for hierarchical-goal generalist action learning across multiple 3D game worlds
The authors present the AceAction dataset, a collection of human gameplay demonstrations across multiple 3D games that links overall task instructions, ordered subgoals and local skill instructions with time-aligned visual observations and keyboard-and-mouse actions, covering navigation, reconnaissance, combat, interaction and recovery. The team uses a vision-language model pipeline to identify subgoals from changes in goals or interactable objects, anchor navigation goals to recognizable waypoints, and compose adjacent subgoals into task instructions. Review of 500 trajectories achieved a 95.8% pass rate in independent evaluation.