Gym-Anything-RT real-time interaction benchmark released
What happened
The paper introduces Gym-Anything-RT, a real-time interaction benchmark for evaluating vision-language agents in real-time settings, and publicly releases the environment along with the generated mini-games. The environment uses an agent pipeline to automatically generate 100 real-time visual mini-games, based on a taxonomy of six real-time interaction capabilities. The paper reports that GPT-6 Astra (xHigh) achieves a pass rate of 9.0% on these real-time mini-games, rising to 75.0% when the environment is paused. The environment can run the same mini-game in real-time, paused, or slowed-down modes, which allows direct comparison of how difficult real-time operation is. The authors also state that some frontier agents, when allowed to write code, bypass real-time perception and control.
AI-written from the coverage · updated 25 d ago
Coverage timeline
Follow the coverage to see the story from different sides.
- arXiv Game AIPickGym-Anything-RT: Scaling real-time environments for vision-language agents
Gym-Anything-RT generates 100 real-time visual mini-games automatically through an agent pipeline to evaluate the real-time interaction abilities of vision-language agents. The paper reports that GPT-6 Astra (xHigh) achieves a pass rate of only 9.0% on the real-time mini-games, rising to 75.0% when the environment is paused. The authors also find that some frontier agents bypass real-time perception and control when allowed to write code. The environment and the generated mini-games are publicly released.