Skip to content
Hot storyPast story

Authors propose AceAction-5M, a cross-3D-game action learning dataset

2 reports1 sourceUpdated 19 d ago

What happened

AI overview

The authors propose and build AceAction (referred to as AceAction-5M in the newly reported title): a vision-language-action dataset for skill composition and subgoal switching, collected from human play across multiple 3D games. It links overall task instructions, ordered subgoals, and local skill instructions with time-aligned visual observations and keyboard-and-mouse actions, covering navigation, scouting, combat, interaction, and recovery. Annotation is done automatically by a vision-language model pipeline: subgoals are identified from changes in goals or interactable objects, navigation targets are anchored to recognizable waypoints, and adjacent subgoals are composed into task instructions. The authors report that a review of 500 trajectories passed independent review at a 95.8% rate; this is the authors' self-reported annotation-quality result, not a validation of downstream task performance.

AI-written from the coverage · updated 18 d ago

Coverage timeline

Follow the coverage to see the story from different sides.

Nov 3
  1. arXiv Game AIPick
    AceAction-5M: A dataset for hierarchical-goal generalist action learning across multiple 3D game worlds

    The authors present the AceAction dataset, a collection of human gameplay demonstrations across multiple 3D games that links overall task instructions, ordered subgoals and local skill instructions with time-aligned visual observations and keyboard-and-mouse actions, covering navigation, reconnaissance, combat, interaction and recovery. The team uses a vision-language model pipeline to identify subgoals from changes in goals or interactable objects, anchor navigation goals to recognizable waypoints, and compose adjacent subgoals into task instructions. Review of 500 trajectories achieved a 95.8% pass rate in independent evaluation.

  2. arXiv Game AIPick
    AceAction-5M: A dataset for hierarchical-goal generalist action learning across multiple 3D game worlds

    The authors present the AceAction dataset, a collection of human gameplay demonstrations across multiple 3D games that links overall task instructions, ordered subgoals and local skill instructions with time-aligned visual observations and keyboard-and-mouse actions, covering navigation, reconnaissance, combat, interaction and recovery. The team uses a vision-language model pipeline to identify subgoals from changes in goals or interactable objects, anchor navigation goals to recognizable waypoints, and compose adjacent subgoals into task instructions. Review of 500 trajectories achieved a 95.8% pass rate in independent evaluation.