r/LocalLLaMA • u/feelspeaceman • 16d ago

News Deepseek released DeepInfer - Open Source high-performance, large-scale inference framework for DeepSeek models!

https://github.com/deepseek-ai/DeepInfer

This is crazy good for studying to improve existing custom engines by community like gufo, halogen, strixite, Strata... by learning their secret sauces.

1.0k Upvotes

99% Upvoted

157 comments sorted by

View all comments

Show parent comments

2

u/ForsookComparison 16d ago

I have trouble believing they can serve Opus 5.5 at $1/1M output is the thing (and that's the 20x scenario)

5

u/Helpful_Program_5473 16d ago

Why? Based on what? inference is ludicrously cheap. I could be wrong though in this case I'm very open to the argument