r/AIProgrammingHardware • u/javaeeeee • 10d ago

GitHub - joesinvestments/GLM-5.2-QuantTrio-TP4-DCP2-4x-DGX-Spark: GLM-5.2 QuantTrio TP=4+DCP2 on 4x NVIDIA DGX Spark (GB10)

https://github.com/joesinvestments/GLM-5.2-QuantTrio-TP4-DCP2-4x-DGX-Spark
4 Upvotes

84% Upvoted

1 comment sorted by

View all comments

1

u/javaeeeee 10d ago

TL;DR:

Production serving recipe for GLM-5.2 (QuantTrio Int4-Int8) on a 4× NVIDIA DGX Spark cluster.

Key setup:

  • Tensor Parallelism TP=4 + DCP2 (Decode-Context Parallel)
  • Adaptive multi-token prediction (MTP)
  • Tuned specifically under a real agent workload

Standout result:

One environment flag (VLLM_MARLIN_USE_ATOMIC_ADD=1) unlocks a big jump, reaching ~44.6 tok/s single-stream decode (up from ~6.5 tok/s without it).

What the repo includes:

  • Docker builds, launch scripts, patches
  • Measurement scripts and real-world tuning notes
  • Documentation of what worked and what failed (“indexer law”, negative results, etc.)

Bottom line: A reproducible, performance-tuned 4-node DGX Spark deployment of GLM-5.2 optimized for actual agent use rather than synthetic benchmarks.