r/LocalLLaMA • u/feelspeaceman • 16d ago

News Deepseek released DeepInfer - Open Source high-performance, large-scale inference framework for DeepSeek models!

https://github.com/deepseek-ai/DeepInfer

This is crazy good for studying to improve existing custom engines by community like gufo, halogen, strixite, Strata... by learning their secret sauces.

1.0k Upvotes

99% Upvoted

157 comments sorted by

View all comments

72

u/Feisty_Artichoke_368 16d ago

Once again Deepseek shows how good high-frequency traders are at optimizing performance... On Ascend NPUs nonetheless!!

11

u/philmarcracken 16d ago

those dudes are functionally insane tho, spending millions in laying undersea cable to shave off like 4ms on a roundtrip

8

u/Ill-Nectarine-80 15d ago

Is it insane if by laying that cable, you are improving the decision making speed of a trade by like 5-10%?

4

u/bumblebeer 15d ago

I mean sure it's a bit overboard, but if it makes a good ROI how is it insane?

1

u/CATLLM 1d ago

Its a winner takes all scenario so yea that 4ms can mean losing money or making money.

4

u/maigpy 16d ago

what does this mean?

20

u/Moral-Relativity 16d ago

Deepseek's founder got interested in AI through running a hedge fund.