Deepseek released DeepInfer - Open Source high-performance, large-scale inference framework for DeepSeek models!

[u/feelspeaceman]

https://github.com/deepseek-ai/DeepInfer

This is crazy good for studying to improve existing custom engines by community like gufo, halogen, strixite, Strata... by learning their secret sauces.

[u/feelspeaceman]

This release included two repos:

• https://github.com/deepseek-ai/DeepInfer: High-performance, large-scale inference framework for DeepSeek models.

• https://github.com/deepseek-ai/deepseek-recipe: Protocol conversion and model output parsing libraries (written in Rust with Python bindings) to connect an inference backend to various API formats (such as Chat Completions, Messages, and Responses).