What and where in identifying and quantifying conventions in humans and AI agents
What and Where in Identifying and Quantifying Conventions in Humans and AI Agents
The paper proposes a convention gap metric in the card game Hanabi to compare how humans and layered agents use conventions. In two independently collected sets of human games, humans have a larger gap than every agent tier. On conventions both sides share, however, humans only match Tier 2, and their advantage is concentrated in clues that point to untouched cards. Convention use relates to players' knowledge of their cards and is most common in the mid-game. Masking several high-gap situations has a much larger effect than the sum of masking each one alone.
The paper turns the convention gap into a measurable metric, which can be used to compare how differently humans and agents of various tiers develop tacit understanding in cooperative games.
Source: arXiv Board & Classic Game AI · arxiv.org