Skip to content
Hot storyPast story

Inworld AI launches KV-cache-aware routing

1 report1 sourceUpdated 2026-10-15

What happened

AI overview

Inworld AI introduces KV-cache-aware routing Inworld AI introduced KV-cache-aware routing on X. Each request is assigned to a replica that already holds its prefix in the KV cache, so shared context does not have to be recomputed. According to Inworld AI, cache hit rates for repeated prompts can reach up to 90%, and the cost of cached input can drop by up to 99%. These are the company's claims, applying to repeated prompts and cached inputs, not a general effect across all requests.

AI-written from the coverage · updated 2026-10-15

Coverage timeline

Follow the coverage to see the story from different sides.

Oct 15
  1. Inworld AI
    Inworld AI reduces redundant computation with KV-cache-aware routing

    KV-cache-aware routing: each request goes to the replica that already holds its prefix in the KV cache, so shared context isn't recomputed. Cache hit rates for repeated prompts can reach up to 90%, and the cost of cached input can drop by up to 99%. Each response is still generated anew.