DeepSeek open-sources DeepInfer inference framework

DeepSeek has released DeepInfer, the framework it uses to serve its own production workloads, on GitHub under the Apache 2.0 license. For now it supports a single model, DeepSeek-V4.1-Flash, and loads the 1017 version by default. The README names vLLM as the project's original foundation.

The framework runs on Nvidia SM100 Blackwell GPUs and Huawei Ascend 950 neural processing units, while Hopper and other older Nvidia architectures are not listed. Two features that keep outputs identical across runs work only on Nvidia hardware. All published benchmarks, which DeepSeek produced itself, ran on Nvidia chips.

Speculative decoding runs through DSpark, where a draft model proposes five tokens at a time, and DeepSeek estimates a 4% to 7% throughput gain at its production load. A second technique runs only the last 128 tokens of each prompt through the decoder layers, which the company says nearly halves total prefill computation. The key-value cache management and load-balancing code has not been released, and splitting prefill and decode across machines is left to operators.

Teams running DeepSeek-V4.1-Flash on Ascend or Blackwell hardware now have a published serving stack to start from, but Hopper-based clusters are not supported. If identical outputs across runs matter for your agents' testing, that depends on Nvidia hardware. Treat DeepSeek's own throughput figures as unverified until independent benchmarks are available.