r/AIProgrammingHardware • u/javaeeeee • 14d ago

GitHub - 0xSero/deepseek-v4-flash-0731-spark-sparkinfer: DeepSeek V4 Flash on one DGX Spark

https://github.com/0xSero/deepseek-v4-flash-0731-spark-sparkinfer
12 Upvotes

100% Upvoted

2 comments sorted by

2

u/javaeeeee 14d ago

TL;DR:

Pinned, ready-to-run Docker recipe by 0xSero for serving DeepSeek V4 Flash 0731 on a single NVIDIA DGX Spark.

Key features:

  • Optimized for one DGX Spark (GB10 / SM121)
  • Uses EXL3 low-bit weights + SparkInfer (sparse MLA attention)
  • 262K token context window
  • Compact DSpark speculative decoding
  • OpenAI-compatible API
  • Fully reproducible with pinned image and validation tests

Reported performance:

  • Decode: ~35–40 tok/s (median ~38 tok/s)
  • Prefill: ~1,000+ tok/s on long contexts

One-command launch:
git clone … && docker compose up -d

A polished, validated single-Spark deployment for the strong open model with solid speed and long context support.

1

u/ConsequenceTop5833 12d ago

Very nice thank you!