RetroChessBench released to test chess history reconstruction
What happened
Tidor-Vlad Pricope proposes RetroChessBench, which uses 12 tasks to test whether models can reconstruct the legal move history of a chess game and distinguish events that could have occurred from events that must have occurred. Across two trials per task, GPT 6 Astra High failed to give the required final answer 22/24 times without tools and 18/24 times with code execution enabled; average terminal RL rewards were 14.75% and 27.19%, respectively.
AI-written from the coverage · updated 23 d ago
Coverage timeline
Follow the coverage to see the story from different sides.
- arXiv Board & Classic Game AIRetroChessBench: A Benchmark for Reconstructing and Proving Chess Histories
RetroChessBench is released, using 12 tasks to test whether models can reconstruct the legal move history of a chess game and distinguish events that could have occurred from events that must have occurred. Across two trials per task, GPT 6 Astra High failed to give the required final answer 22/24 times without tools and 18/24 times with code execution enabled, with average terminal RL rewards of 14.75% and 27.19%, respectively.