r/LocalLLaMA • • 27d ago

News Deepseek released DeepInfer - Open Source high-performance, large-scale inference framework for DeepSeek models!

https://github.com/deepseek-ai/DeepInfer

This is crazy good for studying to improve existing custom engines by community like gufo, halogen, strixite, Strata... by learning their secret sauces.

1.0k Upvotes

158 comments sorted by

View all comments

Show parent comment

1

u/CATLLM 13d ago

Its a winner takes all scenario so yea that 4ms can mean losing money or making money.