DeepSeek open-sources DeepInfer, its production inference engine
DeepSeek released DeepInfer, the inference software behind its own services, on GitHub under Apache 2.0. It targets V4.1-Flash and supports Huawei Ascend chips on par with Nvidia.
What happened
On 20 October DeepSeek released DeepInfer on GitHub under Apache 2.0, describing it as "the high-performance inference framework that powers DeepSeek's large-scale production services." Commercial use, modification and redistribution are allowed with attribution preserved.
What is in it
Per The Decoder and The New Stack, DeepInfer includes optimised kernels, multi-chip distribution, speculative decoding and asynchronous scheduling, plus a deepseek-recipe tokenizer aligned with DeepSeek's API. It targets DeepSeek-V4.1-Flash and supports Nvidia SM100 GPUs and Huawei Ascend 950 NPUs. KV-cache management, cross-machine stage separation and replica load balancing are not yet included.
Why Huawei matters
Huawei gets equal billing with Nvidia. DeepSeek said last week it plans to pre-train on Huawei silicon by the end of 2026, and US export controls limit its access to top Nvidia chips. The same day, the DHS order removing DeepSeek from federal systems was published.
Caveats
No independent performance data has been published, and support beyond V4.1-Flash is unconfirmed. DeepSeek says future models' inference code will join the same repository.
Open-source strategy
DeepSeek was known for releasing open weights; now it is also opening the software that runs them efficiently. Together, the two let companies run DeepSeek models at production quality on their own hardware without paying DeepSeek. That looks like lost revenue in the short term, but building an ecosystem around DeepSeek's software and Huawei hardware fits China's goal of reducing dependence on Nvidia. It also gives developers a reference implementation to compare against widely used open-source serving engines such as vLLM and SGLang. DeepSeek says it wants to help the community "lower compute costs, reduce energy use, and make AI more affordable." Neither OpenAI nor Anthropic had responded publicly.
What it means for your business
- Benchmark costs on your own workload before moving off a cloud API.
- Check regulatory exposure if you sell to US public or enterprise buyers.
- Read model licences separately from the software licence.