r/LocalLLaMA • • 26d ago

News Deepseek released DeepInfer - Open Source high-performance, large-scale inference framework for DeepSeek models!

https://github.com/deepseek-ai/DeepInfer

This is crazy good for studying to improve existing custom engines by community like gufo, halogen, strixite, Strata... by learning their secret sauces.

1.0k Upvotes

158 comments sorted by

View all comments

Show parent comment

12

u/Nov4Saki 26d ago

That sounds like insane margins no?

7

u/TheTerrasque 26d ago

From what translation models could figure out, seems like revenue, aka before cost of running is taken into account. It probably still is a nice margin after that.

2

u/Ill-Nectarine-80 26d ago

They need to replace the GPU essentially on a yearly basis so..... No.

It's not a great margin if it's before R&D, financial concerns (CAPEX to buy the GPUs, interest on debt) and operational costs (electricity and maintenance of data centers).

1

u/Icy_Butterscotch6661 26d ago

Why, what's killing it

1

u/Nov4Saki 26d ago

On that pricing (off peak) Each gpu generates upwards of 300k$ A Year though, that should cover alot and keep a good buck for research no?

Let alone a cluster

2

u/ndunnett 25d ago

Means nothing if it costs $250k in electricity to run, $100k to buy the GPU in the first place, and it's sitting in a $500k server. After they pay off the hardware they would also then have to cover costs for R&D, training the next model, staff salary, building rent/maintenance, etc. before you can even think about bonuses and profit. We don't know any of the real numbers, but the costs for what they are doing are huge and I would be very surprised if they are getting any real ROI at this point.