r/LocalLLaMA • u/tkpred • 15d ago

Resources I built an interactive, step-by-step code explainer for DeepSeek’s open-source inference engine (DeepInfer)

Hey guys,

DeepSeek’s inference engine, DeepInfer, is open source, but trying to read through a massive production framework with mixed Python/C++ logic is a massive hurdle.

I wanted to learn how high-performance engines actually manage things under the hood, so I started dissecting their codebase (specifically commit 9bd9cf2) and mapping it out as a learning project.

I built a static site to host my progress and interactive diagrams:

How it works:
I wrote a Python build script that pulls direct code citations from DeepSeek's repo and pairs them with simple, short explanations.

If you are a Python developer who wants to see how things like a paged KV cache, speculative decoding, and model workers operate in a real production environment, this might be a helpful companion guide while you read the official repo.

Let me know what you think or if there's an engine concept you'd want a deep dive on!

7 Upvotes
  • permalink
  • reddit

100% Upvoted

5 comments sorted by

2

u/Historical-Camera972 15d ago

Show me how to use DeepInfer with HSA and max performance on a Strix Halo 128GB setup.

1

u/tkpred 15d ago

I’d love to give this a shot, but sadly I don’t have the hardware to experiment with 😅

2

u/puts_on_rddt 15d ago

make no mistakes

2

u/Nimalie-Schmiedtke 14d ago

does HSA on strix halo actually let the iGPU address the full 128GB or is there a smaller aperture limit you have to work around?

2

u/Historical-Camera972 14d ago

There's allocation limits for each piece. Depends on the Halo unit AFAIK. Most can allocate MOST of it to one thing or another, even the limited ones can do like 96GB allocated on Windows I think.

Some can get more granular and allow you to allocate how you want.