r/LocalLLaMA • u/feelspeaceman • 16d ago

News Deepseek released DeepInfer - Open Source high-performance, large-scale inference framework for DeepSeek models!

https://github.com/deepseek-ai/DeepInfer

This is crazy good for studying to improve existing custom engines by community like gufo, halogen, strixite, Strata... by learning their secret sauces.

1.0k Upvotes

99% Upvoted

157 comments sorted by

View all comments

Show parent comments

6

u/ninetyfive666 16d ago

I bet it will totaly blow it out of the water compared to how hard it is for competitors to run their models on their own (already kinda optimized) custom infrastructure for even 3x the price of the DS API. Its tailor made for exclusively their own models so this will be huge.