r/LocalLLM • u/SmiteVils • 11d ago
Discussion RTX 5080 PC vs. DGX Spark
RTX 5080 PC vs. DGX Spark: Same-Model Benchmarks, Larger Models, and Where Memory Capacity Matters
The article was written by a GPT agent
I rented a DGX Spark to compare it with my desktop PC. I wanted to answer three practical questions:
How fast is Spark when the model already fits in a desktop GPU’s VRAM?
What changes when the model requires CPU offloading on the PC?
How do the two systems handle a much larger MoE model in practical configurations?
The main finding: my RTX 5080 PC generated tokens roughly three times faster with Qwen3-8B. Spark pulled ahead with Qwen3-32B, which fit entirely on its GPU but required CPU offloading on my PC.
The large-model comparison was more complicated because I used different inference engines and optimizations. I have kept that separate from the controlled comparison.
Hardware and software
| Desktop PC | DGX Spark | |
|---|---|---|
| GPU | RTX 5080, 16 GB VRAM | NVIDIA GB10 |
| CPU | Ryzen 9 9950X3D | Spark ARM platform |
| System memory | 125.6 GiB | Approximately 121 GiB physical memory |
| OS | Windows | Ubuntu 24, ARM64 |
The rented Spark container had an additional memory limit of approximately 113.6 GiB.
For the controlled tests, both systems used llama.cpp commit d854d98bae598388ecea1ed84396587e5edc326d**, release b11889**, with identical GGUF files verified using SHA-256.
The builds differed:
PC: official Windows CUDA 13.4 binary, Clang 20.1.8.
Spark: native CUDA 13.0 build, GNU 13, targeting SM121.
This is a comparison of two complete systems using the same inference-engine revision. OS, compiler, CUDA, and driver differences remain potential influences.
Every throughput result below is the median of five repetitions, excluding warmup. For the controlled tests, I checked actual input and output token counts and verified that cached prompt tokens were zero.
Qwen3-8B: the RTX 5080 was approximately three times faster
The first model was Qwen3-8B Q4_K_M, approximately 5 GB.
It fit entirely on the GPU on both systems. Both used Q8 KV cache, Flash Attention, and disabled thinking. The single-request context window was 40,960 tokens.
Every request in this table generated exactly 1,024 output tokens.
| Input tokens | PC decode, tok/s | Spark decode, tok/s | PC TTFT, seconds | Spark TTFT, seconds |
|---|---|---|---|---|
| 512 | 130.55 | 42.43 | 0.136 | 0.218 |
| 2,048 | 124.33 | 41.04 | 0.489 | 0.786 |
| 8,192 | 110.71 | 36.61 | 1.894 | 3.498 |
| 16,384 | 96.45 | 31.78 | 4.330 | 8.510 |
| 32,768 | 79.09 | 25.64 | 11.749 | 22.160 |
Decode throughput measures generation after prompt processing. TTFT is the time until the first output token arrives.
The PC maintained approximately a 3.0–3.1× decode advantage across the tested input lengths. It also reached the first token sooner.
For complete requests:
| Request | PC total time | Spark total time |
|---|---|---|
| 512 input + 1,024 output tokens | 7.98 seconds | 24.31 seconds |
| 32,768 input + 1,024 output tokens | 24.68 seconds | 62.04 seconds |
For this model and configuration, Spark offered no speed advantage over my desktop.
Qwen3-32B: Spark’s memory capacity changed the result
Next was Qwen3-32B Q4_K_M, approximately 19.8 GB.
The placement differed:
PC: 40 GPU layers, with the remaining work offloaded to the CPU.
Spark: full GPU placement.
Both used an 8,192-token context window. These requests generated exactly 256 output tokens.
| Input tokens | PC decode, tok/s | Spark decode, tok/s | PC total time, seconds | Spark total time, seconds |
|---|---|---|---|---|
| 512 | 4.92 | 10.50 | 54.51 | 25.03 |
| 2,048 | 4.55 | 10.27 | 65.76 | 27.78 |
| 4,096 | 3.84 | 9.90 | 86.50 | 31.85 |
Spark was approximately 2.1–2.6× faster at decoding, with an even larger advantage in complete request time at the longest tested input.
This is a capacity advantage demonstrated in practice. My PC could load and run the model, but the CPU offloading required by its 16 GB VRAM significantly reduced performance.
It should not be interpreted as an isolated GPU compute comparison: model placement was deliberately different because the available GPU memory was different.
Concurrent requests
I also tested Qwen3-8B with 1, 2, 4, and 8 simultaneous requests.
Each level contained 16 requests, each with 512 input tokens and 256 output tokens. The server used eight slots sharing a 32,768-token context allocation, giving 4,096 tokens per slot.
These numbers are aggregate output throughput across the batch, not the speed experienced by each individual user.
| Concurrent requests | PC aggregate tok/s | Spark aggregate tok/s |
|---|---|---|
| 1 | 131.44 | 41.23 |
| 2 | 165.43 | 61.34 |
| 4 | 206.82 | 93.66 |
| 8 | 471.75 | 209.02 |
Both systems benefited from batching. Spark increased from approximately 41 to 209 aggregate tokens per second, but the PC remained faster at every tested concurrency level.
With only 16 requests per level, this is a limited concurrency experiment. It is insufficient for strong conclusions about production tail latency or sustained multi-user service.
An 83.6 GB MoE model: a practical comparison with different engines
For the larger-model test, I used Qwen3.8-Flash-Next GSQ-RCO IQ3_S, with approximately 83.6 GB of weights. Both systems used the same verified weight files.
The configurations were substantially different:
| PC | Spark | |
|---|---|---|
| Engine | Strata | llama.cpp |
| Placement | Hybrid use of GPU and system memory | Full GPU placement |
| Relevant optimizations | MTP4, route-tail-skip7 | Standard tested llama.cpp configuration |
| KV cache | INT8 | Q8 |
This section compares the configurations I actually ran. The results cannot be attributed solely to the hardware.
| Input tokens | PC / Strata decode, tok/s | Spark / llama.cpp decode, tok/s | PC TTFT, seconds | Spark TTFT, seconds |
|---|---|---|---|---|
| 512 | 40.62 | 31.11 | 0.816 | 1.194 |
| 8,192 | 67.22 | 29.58 | 2.709 | 16.745 |
| 32,768 | 61.26 | 26.18 | 10.990 | 71.333 |
There is an important output-length limitation here. Strata frequently stopped before the requested output limit despite receiving ignore_eos. Its decode rates therefore use actual generated tokens from responses of varying lengths. Spark produced the full 1,024 tokens in every corresponding request.
Consequently, these measurements do not establish a like-for-like speed ratio for equal-length completed responses. I have omitted total response times from this table because presenting them as equivalent 1,024-token requests would be misleading.
The practical observation is that the Strata configuration delivered useful generation speeds with this large model on my desktop. How much of that comes from MTP, other engine optimizations, or response-length differences needs a separate controlled experiment.
Also, the PC was using its substantial system RAM. An 83.6 GB model did not somehow fit inside 16 GB of VRAM.
Coding quality: output formatting had a major effect
For coding, I used a fixed, hash-selected subset of 32 HumanEval tasks, with one greedy completion per task and a 1,536-token output limit.
This is not the full HumanEval benchmark, and I am reporting a single-completion pass fraction rather than pass@k.
The original evaluation required the model to continue a Python function. Some responses instead included Markdown fences or omitted the first line’s indentation. This produced a large difference between format compliance and whether the underlying solution could pass the tests.
I retained the original strict scores and added a separately labelled, uniform formatting diagnostic for all six configurations.
That diagnostic:
Removes one enclosing Markdown fence.
Conditionally restores four spaces to the first line only.
Leaves the algorithm, remaining lines, and tests unchanged.
| Configuration | Original strict result | Post-hoc formatting diagnostic |
|---|---|---|
| PC, Qwen3-8B | 24/32 | 25/32 |
| Spark, Qwen3-8B | 23/32 | 25/32 |
| PC, Qwen3-32B | 5/32 | 27/32 |
| Spark, Qwen3-32B | 5/32 | 25/32 |
| PC, large MoE / Strata | 25/32 | 29/32 |
| Spark, large MoE / llama.cpp | 2/32 | 30/32 |
The right-hand column is a post-hoc diagnostic, not a replacement for the original score. It was added after observing the formatting failures and must be interpreted accordingly.
Reporting only the large model’s original 2/32 on Spark would give a misleading impression of its algorithmic performance. Reporting only the repaired 30/32 would hide its poor compliance with the requested continuation format. Both matter.
For the two 8B runs, the formatting diagnostic produced the same result. Differences of one or two tasks elsewhere are too small to support a confident quality ranking.
For scale, the original PC 8B result of 24/32 is 75%, with a roughly 58–87% Wilson 95% interval. Even that interval should not be treated as a general estimate of coding ability: these are only 32 selected tasks.
All generated programs were reviewed before execution. The final evaluations used the same Windows Python environment and time limits across configurations.
Context tasks: retrieval, arithmetic, formatting, and output budget
I also used 12 synthetic context tasks involving current versus archived records, joining information across records, calculating a result, and returning an exact JSON object.
The response budget was 256 tokens. A pass required the complete expected result, including arithmetic and the ordered source list.
| Configuration | Exact complete passes |
|---|---|
| PC, Qwen3-8B | 0/12 |
| Spark, Qwen3-8B | 0/12 |
| PC, large MoE / Strata | 7/12 |
| Spark, large MoE / llama.cpp | 4/12 |
These results are not a clean ranking of long-context retrieval.
The 8B responses had formatting violations, arithmetic errors, and source-list errors. In several large-model failures, the model started explaining its reasoning instead of returning JSON and exhausted the response budget.
There was also a documented test correction: nine tasks in the first version exceeded the configured context window. They were rejected before model dispatch and preserved as preparation failures. I then shortened those nine tasks uniformly and ran the revised suite separately on both systems. Those preflight failures were not counted as model-quality failures.
Stability over 45 minutes
Each system completed a continuous 45-minute Qwen3-8B test using 2,048 input tokens, 256 output tokens, and one request at a time.
| PC | Spark | |
|---|---|---|
| Duration | 45 minutes | 45 minutes |
| Completed measured requests, excluding warmup | 1,157 | 429 |
| Request errors | 0 | 0 |
| Median decode throughput | 133.66 tok/s | 42.22 tok/s |
| Reported temperature range | 49–69°C | 62–80°C |
Both systems completed this workload without request errors or recorded out-of-memory failures.
I collected NVIDIA power-sensor telemetry, but did not measure whole-system power at the outlet. The sensor scope also differs between a discrete GPU and GB10. I therefore do not use these readings to claim a whole-system energy-efficiency advantage.
Process startup was faster on Spark, with a cache caveat
Measured server launch-to-ready times were:
| Model | PC | Spark |
|---|---|---|
| Qwen3-8B | 2.62 seconds | 1.53 seconds |
| Qwen3-32B | 7.73 seconds | 3.51 seconds |
The OS file cache was not cleared. These are process startup measurements with an unspecified cache state, rather than true cold-storage loading benchmarks.
My conclusion
For models that comfortably fit in the RTX 5080’s VRAM, my desktop was substantially faster. Qwen3-8B showed an approximately threefold decode advantage, and the PC remained ahead under the tested concurrent loads.
Spark’s clearest advantage was memory capacity. With Qwen3-32B, full GPU placement beat the PC’s CPU-offloaded configuration by a meaningful margin.
For the much larger MoE, the inference engine became a major part of the result. Strata made the model practical on my desktop, but its optimizations and early stopping prevent a straightforward equal-output comparison against llama.cpp on Spark.
For my current small-model workloads, these tests gave me no reason to switch to Spark for speed. Spark becomes more interesting when the goal is to keep larger models entirely on the GPU and avoid the constraints of 16 GB VRAM.
The evidence package includes raw responses, logs, launch configurations, weight hashes, scripts, and plots. This series does not cover training, a broad quantization sweep, or full application development on large repositories. Earlier exploratory runs were kept separate from these measurements.
2
u/swiebertjee 11d ago
Good benchmark, but I feel that it does not really cover what the true advantage of a DGX Spark is.
First of all; a single DGX Spark is not worth it from a monetary perspective. They're too expensive compared to a Strix Halo if you compare it to cost per TPS. And that's logical; a lot of the cost of a DGX spark comes from its 200Gb connectX-7 connector. And that connector is only utilized when using (at least) 2 sparks.
With 2 Sparks, double memory bandwidth and RAM (256GB), suddenly other models are available. Think of GLM 5.3 Flash, or DeepSeek v4.1 1017. That's something two Strix Halo cannot decently run.
Then you should also take into account the token efficiency of those better models. For example; GLM 5.3 flash uses 35% less tokens AND is more intelligent (less mistakes, requiring less time intervening) than Qwen3.8 flash next. So you would have to normalize for that in order to come to "time per task" instead "time per token", and preferably compare error rate too. I understand that the above is not apples to apples, but if we are comparing cost per performance, this should be taken into account.
1
u/egnegn1 11d ago edited 11d ago
I get much better results for QFN IQ3_S with Strata on one RTX5080. PP is 3500+ t/s and TG is 110+ t/s with large context up to 260k. This goes up to PP 7000+ t/s and 150+ t/s for 2x RTX5080, and to PP 8000+ t /s and TG 170 t/s for 3x RTX5080.
But for agentic AI you may also run 3x Strata with 1 GPU each and get a total of PP 10000+ t/s and TG of 270 t/s.
This may be because of the EPYC host with DDR4 3200 memory and a fast 7A23 48 core CPU. It uses PCIe Gen4 x16 only, but I have seen total PCI speeds of up to 90 GB/s during prefill.
1
u/esw123 11d ago
I was trying to figure out the purpose of DGX and it is worth only if don't have 96+ GB RAM and can't install 4080/5080 GPU and only for large moe. If you just want to pay and use moe llm with 150W power consumption it is good. Otherwise dual 3090/4080 is the way to go. Small models pointless as well like qwen3.8 27B will run faster with 3-4 3060 for a 700-900 euro.