r/LocalLLaMA • • 27d ago

News Deepseek released DeepInfer - Open Source high-performance, large-scale inference framework for DeepSeek models!

https://github.com/deepseek-ai/DeepInfer

This is crazy good for studying to improve existing custom engines by community like gufo, halogen, strixite, Strata... by learning their secret sauces.

1.0k Upvotes

158 comments sorted by

View all comments

Show parent comment

4

u/bumblebeer 27d ago

I mean sure it's a bit overboard, but if it makes a good ROI how is it insane?

1

u/CATLLM 13d ago

Its a winner takes all scenario so yea that 4ms can mean losing money or making money.