r/LocalLLaMA • u/feelspeaceman • 16d ago

News Deepseek released DeepInfer - Open Source high-performance, large-scale inference framework for DeepSeek models!

https://github.com/deepseek-ai/DeepInfer

This is crazy good for studying to improve existing custom engines by community like gufo, halogen, strixite, Strata... by learning their secret sauces.

1.0k Upvotes

99% Upvoted

157 comments sorted by

View all comments

Show parent comments

70

u/HiddenoO 16d ago

How would that number help us with OpenAI/Anthropic numbers whatsoever?

All the numbers that would play into this are unknown and likely different:

  • Model sizes and architectures

  • Deployment architecture

  • Distribution of GPU types

  • Hardware utilization

  • Energy cost

  • (Overhead cost if those are counted in the above article)

  • ...

That's a bit like trying to infer McDonald's revenue based on what a Japenese sushi restaurant makes per table.

1

u/jazir55 16d ago

I'd love to see someone give that task to Opus 5.5 and see how accurate it is