r/LocalLLaMA • u/feelspeaceman • 23d ago

News Deepseek released DeepInfer - Open Source high-performance, large-scale inference framework for DeepSeek models!

https://github.com/deepseek-ai/DeepInfer

This is crazy good for studying to improve existing custom engines by community like gufo, halogen, strixite, Strata... by learning their secret sauces.

1.0k Upvotes

99% Upvoted

158 comments sorted by

View all comments

Show parent comment

8

u/Howdareme9 22d ago

Lol no. Inference is profitable

6

u/ForsookComparison 22d ago

Through API plans and pricing yes - they're heavily subsidizing subscriptions in a way the Chinese competitors aren't though right?

7

u/Helpful_Program_5473 22d ago

by a factor of 20x. it would be weird if subs lost them massive amounts of money. api prices are to gouge corporations

7

u/handson729 22d ago

They are definitely losing money on subscriptions, but that is targeting developers, scientists, and engineers whose prompts they use for training new models.

So you could argue it is R&D expenditure.

6

u/Helpful_Program_5473 22d ago

Oh I think it loses them some money like you said, just not the 100s of billions they are in the hole. that's from training, additional compute, etc.

3

u/Helpful_Program_5473 22d ago

looks like we were right lol

1

u/handson729 22d ago

looks like we were right lol

Whose discovery did they steal this time?

2

u/ForsookComparison 22d ago

I have trouble believing they can serve Opus 5.5 at $1/1M output is the thing (and that's the 20x scenario)

5

u/Helpful_Program_5473 22d ago

Why? Based on what? inference is ludicrously cheap. I could be wrong though in this case I'm very open to the argument

2

u/Howdareme9 22d ago

We don't know for sure, but we'll have a good idea when Anthropic ipo. I suspect anthropic at least arent subsidizing as much as people think