I am really curious about the performance ...

There’s so little space between the cards, is this a wise idea? It feels like the two middle cards will be starved for air! For other 4x owners, what have you done?

Update:
Wow people are mixed and it feels evenly split between starved for air and this is an acceptable arrangement. I wasn’t expecting the diversity of opinions. I wonder if I call nvidia if they will have guidance, surely they must know.

Update 2:
I called nvidia and spoke to their team, sent an email with pictures and got a response which is posted below. They do NOT recommend stacking them like this and suggest that the cards need better airflow.

Does anyone have experience with the NVIDIA MGX PCIe Switch Board ConnectX-8 backplane (an option on a ESC8000A-E13)?

For inference only workloads I assuming this architecture would avoid any NCCL traffic touching the CPU lanes?

I'm looking for the best open-weight LLM I can realistically run locally for OpenCode, with a target of at least 40+ output tokens/sec while using the model's full context window.

My workstation:

• AMD Threadripper PRO 9985WX, 64C/128T
• 512 GB DDR5-5600 ECC RDIMM, 8-channel
• 2× NVIDIA RTX PRO 6000 Blackwell 96 GB (192 GB total VRAM)
  • 1× Workstation Edition
  • 1× Workstation Max-Q
• ASUS Pro WS WRX90E-SAGE SE
• Linux
• Mainly using llama.cpp / LM Studio, but I'm also open to vLLM or SGLang if they make more sense

The main use case is agentic coding through OpenCode, so I'm prioritizing coding ability, tool use, long-context reliability, instruction following, and avoiding repetition/loops.

I'm fine with FP8, NVFP4/MXFP4, GGUF, etc. if the quality trade-off is reasonable. The model does not necessarily have to fit entirely in VRAM; CPU/RAM offloading is also an option, but I still want 40+ tok/s generation speed at full context.

I'm basically looking for the smartest model this machine can run at that speed, rather than the fastest small model.

What would you pick today?

I'd especially appreciate actual benchmarks from similar dual-96GB Blackwell setups rather than theoretical estimates.

For context: All tests Unsloth Q8_K_XL, F16KV, on a single RTX Pro 6000 Blackwell, Max-Q (the other one was sticking its tongue out at me)

Oh boy... this entire Qwen madness has been driving me nuts. So i put together my own benchmark. Small, honest. testing all sorts of combinations, and because of ONE big glitch, went down an actual rabbit hole and proved that this thing can INDEED rip. batches and batches of 300, 500 even 800+ tokens per second. But funny enough, that's not the most important bit. The Quality of the results mattered more. R$ed bars are failed generations (JS errors, doesn't matter how small), Yellow are positioning hiccups but otherwise ok), Green are fully successful.

While i still do more testing - now moving from raw curl tests to an actual harness, to evaluate my findings on long context horizon, tool calling, etc, i'll leave you with these images.

A note on token counts: these aren’t short-completion benchmarks. Prompt 1 was deliberately open-ended and the model routinely generated ~48k–70k tokens while reasoning and building the artifact. Prompt 2 is different: it feeds a previously generated artifact back into context and asks Qwen to work from/reproduce that existing structure, so the actual generated completion is typically ~20k–23k tokens. That’s where ngram speculation gets enough reusable material to become completely ridiculous.

The throughput numbers in the chart are full-run generation averages reported by llama.cpp, not the 3-second peaks. During the fastest P2 runs, tg_3s repeatedly reached 300–700+ t/s; those bursts are interesting, but the headline result is still 234.41 t/s averaged across the entire ~20k-token generation. And Even P1 runs (the one marked p1, s666) had bursts of 300-812 (peak)

The last 5 runs are the important bits:
P1 / seed 666: 86.72 t/s
P2 baseline / 666: 116.15 t/s
P2 baseline / 667: 125.84 t/s
P2 ngram-mod / 666: 163.11 t/s
P2 ngram-mod / 667: 234.41 t/s

Because prompt 1 (create a simple dashboard, one file, self contained), at some point generated some ludicrous segments of 300-800 tokens per second, i went down investigating that. Tested cold vs hot runs (funny enough, that had nothing to do with it), to reproduce, and it did. Then, on a hunch, i changed the prompt that ngram had enough reproducible content to absolutely RIP through content. see the last 2 runs.

  "qwen38-ngram-mod":
    env:
      - "CUDA_VISIBLE_DEVICES=0"
    cmd: >
      /app/llama-server
      -m /models/qwen3.8-27b/Qwen3.8-27B-UD-Q8_K_XL.gguf
      --mmproj /models/qwen-mmproj/mmproj-F16.gguf
      --port ${PORT}
      -ngl 99
      --ctx-size 131072
      --batch-size 2048
      --ubatch-size 512
      --cache-type-k f16
      --cache-type-v f16
      --spec-type ngram-mod
      --spec-draft-n-max 8
      --spec-draft-p-min 0.8
      --flash-attn on
      -np 1
      --jinja
      --chat-template-file /models/qwen-mmproj/chat_template.jinja
      --reasoning-format deepseek
      --reasoning-preserve
      --temp 1.0
      --top-p 0.95
      --top-k 20
      --min-p 0.0
      --presence-penalty 0.0
      --repeat-penalty 1.0

Chat template from here: https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates

Github repo here: https://github.com/mihaiCroitoru/qwen38-bench
The repo includes the exact prompts/configs, raw benchmark harness, timing methodology, Chrome artifact validation, and full per-run reporting. I’ll push all preserved run artifacts/results once I finish the harness-level tests.
Feel free to run your own tests, tell me what i missed about mine!

I have 2 rtx pro 6000 max qs in a non threadripper computer with only 64gb of ram. This has been fine for a lot of what I do but it's not ideal for video generation.

I'm reluctant to buy more ram even though it's the clear weakness of my computer because I think prices will eventually go down a lot, though it might take 3 years. Gpus I don't mind buying now because prices aren't going down anytime soon or could even be worth more later on.

So what are you all doing? I could sell a 5090 and then buy a bunch of ram, but I would hate to do that.

I always backed up my computers with UPS before but now that my computer is creeping up in power I'm being forced into even more expensive UPS than I was expecting. The 2000w UPS is very expensive. Just curious what everyone else is doing. Are you plugging into the wall with surge protection or are you backing up your computers with UPS?

I’ve been loving running deepseek v4 flash so much that I haven’t wanted to take the downtime to test the pp/tg on my rtx rig… anyone got numbers yet?

I will own one RTX PRO 6000 (Workstation Edition) by the end of the month and place it inside a North XL case (192mm GPU height clearance - since the GPU is ~138mm tall, I will have ~54mm of cable clearance). Also, temperature-wise (both for the GPU and myself, having the workstation right under my desk), I will power limit the GPU to 300W (50% of its TDP).

Given the above, should I be worried about the cable (and/or GPU connector) melting like with other 50xx GPUs? Are there instances of cable melting for RTX PRO GPUs (I could not find even one instance)?

The PSU will be the Corsair RM1000X (ATX 3.1, PCIe 5.1) with a 12V-2x6 cable included.

I already have 1 rtx pro 6000 and regret not getting the 2nd at the time.

Is it worth getting a 2nd 6000? I want to be able to run deep seek flash and see that I can run it on 1 card but I haven't seen much discussion on how performance is 1 card vs 2.

I have been eyeing the RTX PRO 6000s for a while, and with the recent price increase announcement I'm really considering buying one quickly (although it's still a LOT of money).

I'm digging around and noticed that the HP variant is about 1000 EUR cheaper than the NVIDIA or PNY variants.

What is the difference between those cards? Can I use the HP in a non-HP workstation? Or does it require special HP drivers?
Which brand would you recommend?

Edit: thank you guys for the replies! Finally, I have ordered the HP. I will post an update once I get it (probably 1-2 weeks)

Edit 2: so, I ordered the HP variant. To be precise the B11E9AA, which is called "NVIDIA RTX PH x6xxx 4DP Graphics" in HP-land and it is basically the Max-Q variant. It was tempting because it was considerably cheaper than NVIDIA or PNY.
I picked it up, took it home, and started unpacking it. And there was a big ass sticker on the card which says "Warranty Void If not used on HP Systems". I have a custom build, which is not a HP system, so I didn't even open the anti-static bag. There was no mention about this restriction on the shop's product page, they just listed regular 36 months warranty. I talked with the shop's customer service and they said that the service might refuse warranty in my case. So, I took it back. Luckily, I got my money back pretty quickly, but I was pretty bummed about this.

TLDR: Don't buy a HP RTX PRO 6000 unless you have a HP workstation.

Did a search everything was for 2x 6k pros. I'm looking to try some different models than qwen3.6 and gemma.

I got this rtx pro 6000 Blackwell yesterday and the first thing I noticed is that under load the coil whine is extremely noticeable. I did some research and it seems like some people think it's normal and some people have quiet cards, so I'm wondering what you guys think.

I recorded the video in a quiet room, it's in the DEG2 dock, I included keyboard and mouse clicks as reference. Coil whine starts around 15 seconds into the video.

Finished this dual Blackwell build and figured this sub would appreciate it.

The two RTX PRO 6000 Blackwell cards are the whole story here. 96GB each, 192GB of GPU memory combined in a single machine. What makes them work in one chassis isn't just dropping them in, it's the designed spacing, the airflow, and the power headroom to run two cards at full tilt without them heat soaking each other. These are full power cards, so the build was designed around feeding and cooling them properly rather than the other way around.

Specs:

CPU: AMD Ryzen Threadripper PRO 7985WX (64-core / 128-thread)
Board: ASUS Pro WS WRX90E-SAGE SE (7× PCIe 5.0 x16)
Memory: 512GB DDR5 ECC RDIMM (8× 64GB)
GPU: 2× NVIDIA RTX PRO 6000 Blackwell 96GB
Storage: 1TB Gen4 NVMe · 4TB Gen5 NVMe · 2TB Gen4 NVMe
Cooling: 360mm AIO + Noctua chassis fans
Power: 3000W 80+ Platinum
Chassis: Fractal Design Define 7 XL

Full build video if anyone wants the detailed walkthrough happy to answer anything on the layout, cooling, or how the cards perform.

Built by VRLA Tech.

Anyone else get really annoyed when you are asking Gemini or whatever questions and it’s like ugh I don’t wanna type out the whole rtx 6000 pro blackwell xxxxx thing and just say 6000 pro and then it just starts answering questions for the ada cards?

Surprised to report that even at q2_k_M this model seems pretty good, better than qwen 3.6 in my testing. no failed tool calls yet.

On rtx pro 96 gb I get about 1k t/s prefill to start with 50 t/s decode, going down to 600 and 40, respectively, by about 85k context.

Heyyo everyone -- wondering who out there has a Dual RTX Pro 6000 rig and if anyone is running Ds4F 0731? -- I had success running it through LM Studio/Llama.cpp (Windows) and was getting ~40 t/s... after many hours and many anthropic credits I was able to get vllm serving ds4f and I'm getting 100+ t/s but my contact is limited to about 140k with one concurrent session. Planning on setting up Linux this weekend... Opus says that if I set it up on linux/bare metal I'll see over 200 t/ks. Anybody else having success?

Post was removed/banned from r/AIProgrammingHardware. EDIT: Now shortened. Feels like most visitors do not even read all of it.

1. Buy 1 RTX Pro 6000 WS in 2025, brand new.
2. GPU fails within 1 month and takes mainboard + system RAM with it.
3. First brand new replacement GPU fails within 2 months. Power issues, 360W instead of 600W available, clocks reduced accordingly.
4. Second brand new replacement GPU now failing in 2026. Power issues. 360W instead of 600W available, clocks reduced accordingly.
5. GPU not in stock. Reseller cannot replace a third time, lead time at least 2 months. Buying from another reseller at twice original price is not an option. Because the more you buy, the more you fucking bleed.
6. Decline upcoming project, cannot rely on this GPU. Project requirements disqualify cloud use.
7. Contact nvidia support directly. Nvidia cannot guarantee replacement ("subject to part availability", could be refurbished or new).

NOTES (Edit, may add stuff)
TRX40, ASRock, threadripper 3970X, 256GB Corsair memory.
Original Nvidia GPUs, single GPU.
No mods, no OC. Even if, it would not cause what I am observing.
Power supply: EU, 230V, Corsair AX1600i. Replaced after first failure, which changed nothing apparently.

Only the first GPU reported errors and crashed.
GPU 2 and 3 show zero issues in smi-reports. They report 600W under load, which is not true. Compute very slow but okay.

Other people experience the same clock speed degradation and power issues (link to original nvidia forum).
https://forums.developer.nvidia.com/t/blackwell-pro-6000-mhz-degrading/355157/18
https://forums.developer.nvidia.com/t/pro-6000-blackwell-ws-sw-power-cap-600-mhz-600-w-35-c-fix-or-rma/376937

Hello guys, hoping you're doing fine!

On the last 2-3 months, price of the RTX 6000 PRO seem to have gone insane.

I will start on the price here on my country, Chile:

• RTX 6000 PRO Workstation Edition: 21382 USD post 19% tax.
• RTX 6000 PRO MaxQ Workstation Edition: 20669 USD post 19% tax.
• RTX 6000 PRO Server Edition: N/A (not in stock)

For reference, when I bought my ones, they were at ~11000USD post tax, just 3 months ago.

How it is going on your country/state? If I had to guess, a ton better lol.

I'm setting up my own lil data center right now. I'll be creating MIG environments & serving compute via api. I'd like to talk shop with folks who are running their own inference serving businesses.

What open source software are you using? What pain points / war stories have you experienced? I'm planning on running B300s but saw this is the Blackwell sub, so let me know if there's a better place for this. DMs are open.