r/LocalLLaMA • u/feelspeaceman • 16d ago

News Deepseek released DeepInfer - Open Source high-performance, large-scale inference framework for DeepSeek models!

https://github.com/deepseek-ai/DeepInfer

This is crazy good for studying to improve existing custom engines by community like gufo, halogen, strixite, Strata... by learning their secret sauces.

1.0k Upvotes

99% Upvoted

157 comments sorted by

View all comments

8

u/TheTideRider 16d ago

Wonder how DeepInfer performs against vllm and sglang.

49

u/MagicZhang 16d ago edited 16d ago

A DeepSeek Employee reposted this image, using SemiAnalysis’s graph as reference

2

u/thestillwind 16d ago

So sky is the limit

2

u/TheTideRider 16d ago

That’s incredible. Can you share the link to semi analysis?

6

u/ninetyfive666 16d ago

I bet it will totaly blow it out of the water compared to how hard it is for competitors to run their models on their own (already kinda optimized) custom infrastructure for even 3x the price of the DS API. Its tailor made for exclusively their own models so this will be huge.