DeepSeek has released DeepInfer, the inference framework it says powers its production services. The public code provides offline inference, while leaving several components of a complete service to operators. Its initial commit is dated October 20, 2026. We reviewed the documentation, without running the engine or reproducing its benchmarks.
What operators get
The release documentation lists DeepSeek-V4.1-Flash as the supported model, with NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs as supported hardware. The code is available under Apache 2.0.
DeepSeek explicitly excludes a service-level KV cache system, separation of vision encoding, prompt processing and token generation across workers, and load balancing across data-parallel replicas. A complete service still needs a KV cache system. Those exclusions are documented requirements, so teams considering deployment should budget for integration work alongside testing the engine.
The offline examples default to the DeepSeek-V4.1-Flash-1017 checkpoint. They can download it through Hugging Face or use an existing local checkpoint directory. Their optional structured-output check validates regex, JSON and grammar-constrained responses, plus normal end-of-sequence termination. Ordinary offline runs leave that check disabled, so generating an answer alone doesn't establish that those checks passed.
Read the throughput accounting before comparing results
The benchmark documentation exposes a qualification worth carrying into any performance comparison. Its decode-throughput metric assumes full acceptance of speculative draft tokens. Speculative decoding proposes tokens ahead of verification, and this metric counts them regardless of their actual acceptance rate. That accounting can differ from the number of accepted output tokens an application receives.
The same guide documents JSONL output containing the configuration and underlying timing quantities. For an evaluation on your own hardware, keep those records and measure accepted output alongside elapsed time. This is our suggested comparison method, rather than a result we obtained from DeepInfer.
The next useful evidence is an independently reproduced run that reports accepted-token throughput and the service components surrounding the engine. DeepSeek's published exclusions give operators a concrete checklist for assessing what that deployment actually includes.