Skip to content
Hot storyPast story

Proposes the MechEval benchmark and the WARC world model

1 report1 sourceUpdated 13 d ago

What happened

AI overview

Paper authors Ku Onoda et al. propose the MechEval benchmark: 69 recurring mechanics are annotated across 20 games in ARC-AGI-3, used to test whether world models can grasp game mechanics they have not seen before. Built alongside the benchmark is WARC, a 35M-parameter vision Transformer world model; the evaluation metric counts the number of interactions it needs to predict the effects of mechanics in a new environment.

AI-written from the coverage · updated 12 d ago

Coverage timeline

Follow the coverage to see the story from different sides.

Nov 9
  1. arXiv Game World Models & GenerationPick
    How many interactions does 35M-parameter WARC need to learn unseen game mechanics on ARC-AGI-3

    The authors introduce MechEval on ARC-AGI-3, labeling 69 recurring mechanics across 20 games, and build WARC, a 35M-parameter vision Transformer world model, to examine how many interactions it needs to predict the effects of mechanics in a new environment.