What was announced
Sarvam has opened its inference platform, previously previewed as Sarvam Inference at its Epoch event, under the name Sarvam Token Factory. It is a managed inference stack rather than a single model, and developers reach every model through one OpenAI-compatible API.
The launch lists 8 models: Sarvam 105B (105B total, 9B active parameters, 128K context), GLM 5.3 and GLM 5.3 Flash, DeepSeek V4.1 Flash 1017, DeepSeek V4 Pro, Kimi K3 and K3.1 Pro, and Gemma 4 31B.
Infrastructure and how it works
The models run on NVIDIA B200 clusters in India. According to Sarvam, Sarvam 105B now fits on a single B200 per replica, and on-premise deployment is offered to enterprises.
The stack has three layers: a cache-aware control plane that routes requests, separate GPU pools for prefill and decode, and a cache layer combining GPU memory with a shared store. It builds on open source components including vLLM, SGLang, Dynamo and Mooncake.
Key numbers
Sarvam's self-reported, not independently verified figures include:
- DeepSeek V4.1 Flash serves more than 450B tokens per day, with 95.7% of prompt tokens served from cache.
- On GLM 5.3, the planner cut prefill GPU-hours from 32.3 to 17.1 under the same load.
- Sarvam 105B delivers a 0.2-second first token and 330 tokens/s for a single user.
- Kimi K3.1 Pro reaches 6,177 tokens/s per GPU across 16 B200s, versus 3,213 for the compared provider.
Not a clean sweep
Sarvam acknowledges the results are mixed. On DeepSeek V4.1 Flash with a 10K prompt, Together posts a 0.71-second first token versus Sarvam's 0.90 seconds, and streams faster at 278 tokens/s versus 130 on Sarvam Standard.