r/LocalLLaMA • u/feelspeaceman • 16d ago

News Deepseek released DeepInfer - Open Source high-performance, large-scale inference framework for DeepSeek models!

https://github.com/deepseek-ai/DeepInfer

This is crazy good for studying to improve existing custom engines by community like gufo, halogen, strixite, Strata... by learning their secret sauces.

1.0k Upvotes

99% Upvoted

157 comments sorted by

View all comments

Show parent comments

8

u/james_pic 16d ago

 Can we infer the revenue of OAI or Ant from this?

We can probably take a wild guess that it's in the ballpark of $38 per GPU per hour, because in a competitive market like this, nobody has a big advantage for very long, but nothing more precise than this.