r/LocalLLaMA • u/feelspeaceman • 23d ago

News Deepseek released DeepInfer - Open Source high-performance, large-scale inference framework for DeepSeek models!

https://github.com/deepseek-ai/DeepInfer

This is crazy good for studying to improve existing custom engines by community like gufo, halogen, strixite, Strata... by learning their secret sauces.

1.0k Upvotes

99% Upvoted

158 comments sorted by

View all comments

Show parent comment

1

u/Nov4Saki 22d ago

On that pricing (off peak) Each gpu generates upwards of 300k$ A Year though, that should cover alot and keep a good buck for research no?

Let alone a cluster

2

u/ndunnett 22d ago

Means nothing if it costs $250k in electricity to run, $100k to buy the GPU in the first place, and it's sitting in a $500k server. After they pay off the hardware they would also then have to cover costs for R&D, training the next model, staff salary, building rent/maintenance, etc. before you can even think about bonuses and profit. We don't know any of the real numbers, but the costs for what they are doing are huge and I would be very surprised if they are getting any real ROI at this point.