Skip to content
arXiv Game AI· Hyunwoo Ryu·· 2026-10-17PickAI score45

Mirror: self-improving video-to-action models

Self-Improving Video-to-Action Models

AI summary

Mirror is a self-supervised method that learns video-to-action mapping through interaction with the environment, without prior action data or hand-written rewards. On Atari, OGBench, and DeepMind Control, it outperforms mainstream observation-based learning methods using its learned video plans. On Minecraft BASALT, it surpasses methods that have access to expert actions or hand-crafted task knowledge. In Minecraft survival tasks, it obtained an iron pickaxe through self-improvement alone, with no reward or expert action labels.

Why it matters

It doesn't rely on human action annotations or hand-written rewards, letting video planning models become executable agent policies. Worth a look for readers interested in combining video models with agents.

Source: arXiv Game AI · arxiv.org