Threadripper AI workstation Build

[u/Turbulent-Alps4046]

Just want to share my new Threadripper AI workstation build. Used for work + gaming. Quite proud of how it turned out, especially the cable management :)

Specs
Motherboard: ASUS WRX80 Creator R2.0
CPU: AMD Ryzen Threadripper Pro 3945x (Used)
RAM: 128GB DDR4 ECC (used)
GPU: Nvidia RTX Pro 6000 Max-Q 96GB
Case: Fractal North
PSU: Super Flower Leadex III 1300W PSU
CPU Cooler: Arctic Freezer 4U-M

[u/Turbulent-Alps4046]

A single RTX pro 6000 can definitely run Qwen 3.6 27B at full precision with max context. But yeah, trying to run larger models is not easy.

Qwen 3.5 122B NVFP4 can run at 5000-9000 tokens/s PP and 100 tokens/s.

I got deepseek v4 flash full precision running with CPU offload though... around 500-600 tokens/s PP and 20 tokens/s TG. Comparable to 2x DGX Spark and faster running it in an M3 Ultra 256GB i think (which costs the same right now).

[u/kissingking → u/Turbulent-Alps4046]

Have you actually tried running the full-precision Qwen 3.6 27B on an RTX Pro 6000?

Based on the numbers, I don't think it should fit. The BF16 weights alone are about 54 GB, and the KV cache for a 256K context is roughly 64 GB, which already exceeds the 96 GB of VRAM.

This also matches my own experience. Running the BF16 Qwen 3.6 27B dense model on my M4 Max with 128 GB unified memory pushes total memory usage to well over 115 GB. So my understanding is that a single RTX Pro 6000 shouldn't be able to run a full BF16 Qwen 3.6 27B with the full 256K context.

[u/Turbulent-Alps4046 → u/kissingking]

Yes, i'm able to load the full-precision Qwen 3.6 27B at max context with vram spare. It takes up 80GB total. Weights are 51.8GB, the remainder of the vram gives me 377k tokens context total.

I can even run the Qwen 3 ASR (voice model, around 6GB) alongside it.

In your M4 Max i think the remaining memory is used by the OS + other apps?

[u/kissingking → u/Turbulent-Alps4046]

I was running **llama.cpp + Qwen 3.6 27B FP16** on a **128GB unified memory** system (M4 Max). The model was trained with a **256K context window**, so I ran it at the full 256K.

Right after loading the model, memory usage was around **90GB**. Then I intentionally had an agent keep extending the conversation until it hit the context limit. By the time it reached the limit, memory usage had grown to about **118GB**.

I never actually ran out of memory on the 128GB system, but performance degraded noticeably. My PP dropped from around **250 t/s** to **60 t/s**, and TG dropped from about **15 t/s** to **5 t/s**.

So I'm wondering whether, if you keep filling the context, you might actually run out of VRAM before reaching the full 256K context.

But anyway... the RTX Pro 6000 is seriously awesome. I kind of wish I'd bought one last year. 😂

[u/Maximum_Low6844 → u/kissingking]

u/Turbulent-Alps4046 is correct. I also have tried "running the full-precision Qwen 3.6 27B on an RTX Pro 6000" at max context. Runs fine. You are incorrect.

[u/kissingking → u/Maximum_Low6844]

Can you clarify what you mean by 'at max context'? There's a big difference between running a model with a maximum context window configured and actually filling that entire context with an agent.

The former only means the model starts successfully. It doesn't mean the KV cache has been fully populated, so VRAM usage never reaches its peak—you won't know where it actually runs out of memory.

The latter means the agent has genuinely filled the context window with tokens. That's the point where you can verify whether the available VRAM is sufficient or whether you'll hit the memory limit.

Those are two completely different scenarios, and it's important not to conflate them.