Explaining chess and Go in natural language, with the BinProp method
Explaining Chess and Go in Natural Language
The paper proposes an automatic metric based on chess engine evaluation and finds that current mainstream large language models perform markedly poorly at explaining chess moves, often relying on surface features of the board. The proposed BinProp is a training-free method that lets such models discover reusable chess concepts; its average score is nearly triple that of direct prompting. The same method also discovers useful concepts in Go, checkers, and shogi, without additional tuning.
The method needs no additional training yet lets a language model discover reusable chess concepts, offering a useful reference for approaches to explaining engine moves.
Source: arXiv Board & Classic Game AI · arxiv.org