Skip to content
Hot storyPast story

Inworld AI rebuilds model inference stack, claims major performance gains across models

1 report1 sourceUpdated 2026-10-15

What happened

AI overview

Inworld AI says it rebuilt model inference, with large gains in several models' performance Inworld AI said on X that it has rebuilt the online serving approach for each model. The changes mainly include tuning a speculative decoding draft model separately for each model, using custom attention kernels and fused MoE kernels, and optimizing tail latency. These performance results are the company's own claims. The company's figures each apply to a specific model and metric: custom attention kernels made Gemma 4 26B verification up to 7.5x faster; fused MoE kernels made MiMo 2.6 Pro's time to first token 3x faster; tail-latency optimization cut DS V4 Flash's p99 time to first token by about 10x. These numbers cannot be directly generalized to other models or scenarios.

AI-written from the coverage · updated 2026-10-15

Coverage timeline

Follow the coverage to see the story from different sides.

Oct 15
  1. Inworld AI
    Inworld rebuilds model inference, with major performance gains

    We rebuilt how each model is served: - Speculative decoding, with draft models tuned for each model - Custom attention kernels: verification up to 7.5x faster (Gemma 4 26B) - Fused MoE kernels: 3x faster time to first token (MiMo 2.6 Pro) - Tail-latency optimization: p99 time to first token about 10x faster (DS V4 Flash)