Skip to content
arXiv Game AI· Hengguang Zhou·· 3 d agoAI score38

How to evaluate user simulators for interactive agents

On Evaluating User Simulators for Interactive Agents

AI summary

Existing user simulator evaluation frameworks struggle to reliably detect meaningful failures in simulated user behavior: the evaluation methods of three prior studies detected only 5-6 of 10 human-crafted perturbations, and their combined scores even favored simulated user populations instructed to accept incomplete work.

Source: arXiv Game AI · arxiv.org