Inworld AI launches KV-cache-aware routing
What happened
Inworld AI introduces KV-cache-aware routing Inworld AI introduced KV-cache-aware routing on X. Each request is assigned to a replica that already holds its prefix in the KV cache, so shared context does not have to be recomputed. According to Inworld AI, cache hit rates for repeated prompts can reach up to 90%, and the cost of cached input can drop by up to 99%. These are the company's claims, applying to repeated prompts and cached inputs, not a general effect across all requests.
AI-written from the coverage · updated 2026-10-15
Coverage timeline
Follow the coverage to see the story from different sides.
- Inworld AIInworld AI reduces redundant computation with KV-cache-aware routing
KV-cache-aware routing: each request goes to the replica that already holds its prefix in the KV cache, so shared context isn't recomputed. Cache hit rates for repeated prompts can reach up to 90%, and the cost of cached input can drop by up to 99%. Each response is still generated anew.