Skip to content
Sıfırıncı Dakika
Latest

Sarvam opens Token Factory, serving 8 open models from India via one API

AILaunch1 min read

In brief

Sarvam has opened Sarvam Token Factory to everyone, an inference platform that serves its own and open-weight models from NVIDIA B200 clusters in India. Developers reach all 8 models through a single OpenAI-compatible API, aimed at low-latency inference without routing traffic to overseas clouds. All speed figures are Sarvam's own measurements.

  • 8 models via a single OpenAI-compatible API
  • Hosted on NVIDIA B200 clusters in India
  • Sarvam 105B fits on one B200 per replica
  • On-premise deployment for enterprises
  • Three-tier inference stack built around caching

What was announced

Sarvam has opened its inference platform, previously previewed as Sarvam Inference at its Epoch event, under the name Sarvam Token Factory. It is a managed inference stack rather than a single model, and developers reach every model through one OpenAI-compatible API.

The launch lists 8 models: Sarvam 105B (105B total, 9B active parameters, 128K context), GLM 5.3 and GLM 5.3 Flash, DeepSeek V4.1 Flash 1017, DeepSeek V4 Pro, Kimi K3 and K3.1 Pro, and Gemma 4 31B.

Infrastructure and how it works

The models run on NVIDIA B200 clusters in India. According to Sarvam, Sarvam 105B now fits on a single B200 per replica, and on-premise deployment is offered to enterprises.

The stack has three layers: a cache-aware control plane that routes requests, separate GPU pools for prefill and decode, and a cache layer combining GPU memory with a shared store. It builds on open source components including vLLM, SGLang, Dynamo and Mooncake.

Key numbers

Sarvam's self-reported, not independently verified figures include:

  • DeepSeek V4.1 Flash serves more than 450B tokens per day, with 95.7% of prompt tokens served from cache.
  • On GLM 5.3, the planner cut prefill GPU-hours from 32.3 to 17.1 under the same load.
  • Sarvam 105B delivers a 0.2-second first token and 330 tokens/s for a single user.
  • Kimi K3.1 Pro reaches 6,177 tokens/s per GPU across 16 B200s, versus 3,213 for the compared provider.

Not a clean sweep

Sarvam acknowledges the results are mixed. On DeepSeek V4.1 Flash with a 10K prompt, Together posts a 0.71-second first token versus Sarvam's 0.90 seconds, and streams faster at 278 tokens/s versus 130 on Sarvam Standard.

Why it matters

It shows how regional inference options are multiplying for India-based teams and those mindful of data residency, though the speed claims are Sarvam's own measurements.

Sources

Related stories