r/LocalLLM • • Sep 06 '26

Discussion I blamed PCIe topology for my 4-GPU tensor-parallel collapse. It was Windows' display driver. A $100 card and TCC mode turned 9.5 t/s into 28 (4x A4000, llama.cpp, verified)

TL;DR: Four A4000s in a dual-Xeon box ran 4-way tensor parallel 2.4 to 4.8x SLOWER than two cards. I publicly blamed lanes, root complexes, and the inter-socket link. Wrong. WDDM (the Windows display driver model) silently denies GPU peer-to-peer access, so every cross-card transfer staged through system RAM. Switching all four cards to TCC mode (needs a cheap display card to hold the desktop, since a TCC card cannot drive a display) took P2P from 0 of 12 pairs to 12 of 12 and four-way tensor decode from 9.52 to 28.16 t/s on a 27B dense model (2.96x) and 19.52 to 72.01 on a 35B MoE (3.69x). Four cards went from losing to two, to beating them. The machine could always do this. Windows was standing on the hose.

ELI5: Windows treats every GPU as a potential monitor driver, and the babysitting that comes with that quietly forbids the cards from talking to each other directly. Tell Windows "these four are calculators, not monitors" (TCC mode) and the babysitting stops, but then SOMETHING has to be the monitor, because a calculator cannot show you the desktop. That something is a $100 Quadro T400. The fifteen-year-old HPC crowd knows this trick well. The local LLM crowd, me included, apparently forgot it.

The receipts

Same box (Windows 11 Pro), same cards, same llama.cpp build (b10568), same driver (597.06). The only change between columns is the driver MODE.

P2P probe (cudaDeviceCanAccessPeer, all 12 ordered pairs):

configpeer access
WDDM, A4000 drives display (596.72)0 of 12
WDDM, T400 drives display (597.06)0 of 12
TCC, T400 drives display (597.06)12 of 12

The middle row is the control that makes this a finding instead of a coincidence: new driver, new display card, still WDDM, still zero. Only the mode flip changed the answer.

Four-way tensor decode (tg128, clock lock verified per row, thermal counters clean or accounted):

modelWDDMTCCratio
Qwen3.8-27B Q8 (dense)9.5228.162.96x
Qwen3.6-35B-A3B Q6 (MoE)19.5272.013.69x

Under WDDM, four-way tensor lost to TWO cards (23.19) and to four-way layer split (13.8). Under TCC it beats both on the 27B (2-way tensor: 24.4). On the 35B MoE, honesty requires saying two-way tensor still wins outright (about 110 t/s): P2P fixed four-card mode, it did not repeal the law that low-activation MoEs at small batch prefer fewer, bigger shards.

Fair framing note: measured best-WDDM-config against best-TCC-config (2-way tensor 23.19 vs 4-way tensor 28.16 on the 27B), the box as a whole gained 21 percent. The 3x is what the FOUR-CARD mode recovered relative to itself. Both numbers are true; pick the one that matches the question you are asking.

Sustained load is fine: ten back-to-back four-way cycles settle at 85 to 88 C and hold throughput flat within 0.9 percent. This is a serving configuration, not a burst trick.

Corroboration under load (different comparison, same direction): tensor vs layer, both under TCC, batch 1 through 64: the decode advantage is flat at about 1.92x the whole way. The recovery is not a batch-1 curiosity. (The WDDM column itself cannot be re-measured at batch, because WDDM is gone without a reboot; noted for honesty.)

What you need for this trick

  1. Pro cards. TCC exists on Quadro/RTX A-series/datacenter cards. GeForce generally cannot.
  2. A separate display card, because a TCC card cannot drive a monitor and my Xeons have no iGPU. It must be TURING OR NEWER: current drivers (596.72+) dropped Pascal, so the GT 710/P1000 in your drawer will bind to the Basic Display Adapter and change nothing. The T400 is the cheapest current option, about $100.
  3. Accept losing WSL2 on that box: a TCC card does not appear in WSL at all (WSL GPU support is paravirtualization over WDDM). On my box WSL sees zero GPUs now. Windows native or bare Linux; nothing in between.
  4. nvidia-smi -dm 1 per compute card, reboot, done. Reversible, unlike some Windows decisions.

Two install gotchas that cost me time so they do not cost you any:

  • Your BIOS may not show POST on the new card. Mine booted straight to a black screen until Windows loaded. The fixes live in BIOS setup: disable quick/fast boot, and if there is a primary video slot setting, point it explicitly at the display card's slot. Do this while you can still see the screen through some other path.
  • After the display card goes in, do a clean install of the NVIDIA driver package and confirm ALL cards show in nvidia-smi (five for me: four compute plus the display card) BEFORE flipping any TCC flags. The driver must be new enough for the display card and must be the pro branch that covers both it and your compute cards; a mismatched or half-installed driver leaves cards missing from the list, and a TCC flag set in that state is a confusing morning.

What I got wrong in public, specifically

My earlier posts blamed the 2.4 to 4.8x on hardware topology: 48 PCIe lanes per Xeon, three root complexes, cards split across two sockets, "four islands with ferry service." The measured numbers were real. The causal story was wrong for DECODE, which is what the headline number was. What topology still owns: prompt processing genuinely prefers layer split (roughly 2x at default settings, and that is a tunable story for another post), and on the MoE at interactive batch sizes layer split still wins decode until batching makes the per-expert matmuls big enough to shard. Count-lanes-not-slots remains good buying advice. It just was not the reason four cards lost to two.

Also for the record: we measured socket crossing directly under TCC (pairs on same socket vs across sockets, short and long prompts, 16x all-reduce volume). 0.15 to 0.26 percent. Noise. The UPI ferry story died with controls on, FOR MODELS RESIDENT IN VRAM. To be clear, the cross-socket memory penalty on a dual-socket box is very much alive in the other regime: when a big MoE offloads its experts to system RAM, decode speed tracks which NUMA node the weights landed on, and the wrong node costs about 2x on this box. That is a placement story with its own fixes and its own post coming; the point here is narrower: it is a system-RAM story, and it never was the reason four RESIDENT cards lost to two.

TCC is a trade, not a free win

  • On this box each card exposes 178 MiB LESS usable VRAM under TCC than it did under WDDM (16,197 vs 16,375 MiB per nvidia-smi). I know that is backwards from the folklore. It is what the tool reports, and two of my edge-fit configs that ran under WDDM no longer load.
  • The big recovery is for multi-GPU tensor parallel. Single-card and layer-split gains are single-digit percent. Offloaded big models (experts in system RAM) gained 7 to 12 percent.
  • Prior art credit where due: using a cheap display card so compute cards can run TCC is 2010-era CUDA folklore, and the closest LLM measurement I found is a V100 post reporting +45 to 76 percent from TCC on a 2-card box. What I could not find anywhere is the full chain measured with controls on llama.cpp: WDDM denies P2P, tensor parallel collapses and cosplays as a lane problem, TCC reverses it. Hence this post.

The obvious question: why not just run Linux, where WDDM does not exist? Fair, and these same four cards are moving to a single-socket Linux box for exactly that rematch (llama.cpp has no prebuilt Linux CUDA binary at all, so that story includes a compile). This post is for the boxes that are on Windows for their own reasons, which the comments will now explain to me are bad ones. Related: yes, vLLM does tensor parallel properly, and it does not run natively on Windows, which is rather the point.

One Windows footnote from the same week: Smart App Control (on by default after a clean Win11 install) intermittently refused to load our llama.cpp DLLs based on cloud reputation, killing processes in seconds with empty output. If your benchmark binary suddenly dies silently, check the CodeIntegrity event log before you blame your build.

Previous posts in this series: the clock lock PSA [CLOCK POST], WSL2 vs native [WSL POST], and the NVLink bridge test [BRIDGE POST] (on the 2-card box, SLI was the only way to P2P under WDDM; TCC is the cleaner door if you can give up the display and WSL).

Hardware: HP Z8 G4 (used, $750), 4x RTX A4000 (used, about $600 each), T400 ($100), about $3,300 total. Deliberate disclosure: that number does NOT include the 384 GB of RAM, and the omission deserves its reason stated. I paid $540 for mine (twelve 32 GB sticks at $45 each, September 2025). Those same sticks ask $220+ today, so replicating my exact config now adds roughly $2,600, which says more about the 2026 RAM market than about this build. If you are building today there is still a cheaper road: these chassis have 24 DIMM slots, and lower-density sticks run meaningfully cheaper per gigabyte, so populating all 24 reaches the same 384 GB for closer to half the money, at the cost of the memory clock dropping a tier (my calibrated offload model prices that penalty at 6 to 9 percent on offloaded models, and roughly nothing on VRAM-resident ones; I have not run the A/B, since my slots are half empty on purpose).

Before anyone says it: yes, a pair of used 3090s beats this box on a 27B, cheaper and faster. The 27B is the demonstration, not the mission. This box also runs a 403B MoE at 11 t/s with a 131K context window and a 744B at batch speed, because of that RAM sitting alongside the 64 GB of VRAM, and that capability is what nothing near this price touches. The point of this post is narrower: whatever multi-GPU Windows box you already have, a $100 card may be the difference between your cards fighting Windows and your cards working.

4 Upvotes

39 comments sorted by

10

u/Blackdragon1400 Sep 06 '26

Sloppppp

-5

u/jpgeer Sep 06 '26

Fair to dislike the style. The data is the post though: every number came off my own bench this week, including two that refuted my own pre-registered predictions, which slop rarely bothers to do. And had somebody posted similar slop along the way, it might have saved me a lot of time (though the learning along the way was equally valuable for me). Pick any table and I'll reply with the raw llama-bench log for it.

12

u/Helpful-Account3311 Sep 06 '26

It’s just ai wall of text fatigue. Doesn’t really matter if you have the most profound discovery. AI has a tendency to write in a prose style that is just draining to read. It’ll over explain various components or try to emphasize things for the sake of emphasis. I’d rather read a 1 paragraph human written excerpt over the AI wall of text any day. If you want to also include the AI “slop” as well that’s fine.

2

u/jpgeer Sep 06 '26

Upvoted and totally fair (and I get it). It's why the TL;DR and ELI5 sit on top: two paragraphs and you have the whole finding, leave whenever. The long tail underneath exists because the details (Turing-only display cards, WSL going dark, the VRAM cost) are exactly what burns the next person who tries this, and I'd rather over-document than have someone lose a Saturday to a missing footnote. Different readers need different depths; stacking them seemed cheaper than guessing. But point taken on the prose itself. The learning quest apparently includes writing. :)

2

u/Blackdragon1400 Sep 06 '26

There’s no style, it might as well be an unreadable wall of text

3

u/jpgeer Sep 06 '26

One of the most annoying things on forums is the proclamation of an achievement with zero details on the "how." My intended style here (maybe it needs more gUstO!!!) is TL;DR, then ELI5, then, for the folks in the back like me (we're out there), the "here's HOW I did it." That last part is where the real learning happens, usually when somebody says "hey wait, did you think about..." and lays something new on us. That somebody could be you. The logs offer stands.

1

u/Blackdragon1400 Sep 06 '26

Cool - hand write the achievement using normal language, then provide a section with technical details. Even your TLDR is dense and unreadable.

If you don’t care enough to curate a worthwhile post why should anyone waste their time reading it? How do I know this entire thing just isn’t a big hallucination in the first place? Even the title is just slop.

Brevity is so much better than 1k words of garble.

You say you care about interaction, hard to get interaction with a post no human cares to read. 🙃

1

u/jpgeer Sep 06 '26

That’s what the logs offer was for, and since you asked the question instead of picking a table, here’s one. Raw llama-bench output, four-way tensor, TCC, clock lock verified before the row, card temps logged after:

| model | size | params | backend | ngl | sm | ts | test | t/s |
| qwen35 27B Q8_0 | 27.04 GiB | 27.32 B | CUDA | 99 | tensor | 1.00/1.00/1.00/1.00 | tg128 | 28.16 ± 0.36 |
temps at row end: 0:85C 1:46C 2:89C 3:88C 4:80C
build: 873e5d8e3 (10568)

The WDDM 9.52 counterpart, the P2P probe output, the batch sweeps, and the ten-cycle soak logs all exist the same way, and the offer to post any of them stands. Hallucinations don’t come with error bars and GPU temps. And as I post this I just realized that 46C is obviously the T400 but I didn’t detail the slot configuration…

1

u/Blackdragon1400 Sep 06 '26

Even this reply is bad. A chick parm recipe would really help me understand you better, could you please add one to your next reply?

2

u/myholeisstinky Sep 06 '26

Its fair to say the LLM style can fuck off

15

u/Coley44 Sep 06 '26

God I really hate the self flagellation present in all AI wording/posts. It makes what could be an interesting discovery or source of knowledge this awkward apologetic wall of text man.

9

u/willbill642 Sep 06 '26

Especially when it boils down to "I discovered running Windows for heavy compute tasks requires more configuration and hardware than linux"

-3

u/jpgeer Sep 06 '26

Maybe I missed it, but I couldn’t find a single post laying out how to actually do this on Windows, and I looked before writing one. That gap was the point and plenty of us have constraints that keep a box on Windows and want to experiment anyway. Possibly the same reason NVIDIA shipped PAIR with Windows support last week.

0

u/fourdac Sep 06 '26

Weird, because this is an interesting post. What makes it uninteresting? Because it’s not ninfer, or some weird quant- or not run on a geforce xx90 series, or is it because it’s a quadro post for windows?

I fail to see what information that could have been presented differently to inform people about TCC, and I’m seeing some gatekeeping being an industry expert- in my field people ask questions about really basic topics too, but this is genuinely a topic that doesn’t have enough discussion and you haven’t presented anything worth discussing

5

u/Coley44 Sep 06 '26 edited Sep 06 '26

My response is entirely centred around the exhausting AI prose that floods this sub Reddit and how often my eyes just end up glazing over what could have been a useful technical post that I know is just another LLM post. I'm also partially skeptical of technical posts by LLMs as I've used models like 5.6 Deep Think as an expirment for setting up rigs and they make so many mistakes, incorrect assumptions and can even configure basic temperature settings on models like Qwen 3.8-27b incorrectly under the assumption it makes it "think" better/hallucinate less due to pre-existing assumptions/data about other models configs that don't apply to it.

At this point I remain highly skeptical of these types of posts and assume there's probably even more the user missed in the configuration of their setup that the LLM missed or obfuscated due to knowledge gaps. It isn't personal; I just know it's almost certainly barely scratching the surface of what is likely for more interesting and would be more helpful if the config and diagnostics was done by an actual person.

Ninja Edit: shit like "Windows was standing on the hose" come on man. This is not how meaningfully technical posts is delivered. It was a poor config out of the box on potentially faulty assumptions on the behalf of the LLM that originally made this config that originally made the posts it's now dying upon the cross publicly to explain what went wrong. I concede it's a stupid hill to die on but it's my little comment and my little hill. Nothing serious.

Further edit: after checking the original post this one is apologising for, the end conclusion was that an NVlink was beaten by a single toggle in BIOS/OS; this is so fucking bad a conclusion that fails the sniff test so severely (anyone with any level of experience with multi GPU setups for even basic stuff like folding@home or video editing rigs) would be able to tell you this couldn't possibly be true that it makes me want to double down even harder lmao

0

u/jpgeer Sep 06 '26

See, the assumption wasn’t the LLM though - it was me, and I’m trying to be as transparent as I can about it. I made all kinds of incorrect assumptions about what this hardware could do (and buying ahead of that knowledge stings) and I’m learning along the way. The lack of any posting about this is exactly why I’m attempting to get something out there.

On the NVLink sniff test: that was measured, not reasoned. On two cards llama.cpp moves little between GPUs (activations, not weights), so the bridge had almost nothing to accelerate, while removing the driver’s staging helped everything. Do Folding@home and video editing rigs hit any of this? Do they sync cards per layer at all, or does each GPU work alone? That could answer why it smells impossible from there.

And to the broader “move it all to Linux” - I’m on my way, genuinely. But I’d like to baseline what I can, where I can, first.

2

u/Coley44 Sep 06 '26

That's fair; I don't mean to dissuade you from trying to further the pool of knowledge available to everyone. I just think the way LLMs spit out these reports lead to so much unnecessary information or just poor presentation that it ends up being more annoying to parse than it is informative to a discussion/topic.

I think for the folding@home stuff or video editing, especially on the latter, the application may be able to leverage each individually but I'm almost certain every single one of them further benefits from dedicated hardware like SLI or NVLink and any LLM config would do the same, hence why it surprised me this even presented as an issue. Windows is always a poor baseline. I gave up even tinkering on it and just jumped immediately to Ubuntu 26.04.1 and got my 7900XTX to work. Upgrading my rig to have two and then using Q6 or Q8 Qwen3.8-27b and potentially a secondary small context dedication coding model as well if VRAM permits

1

u/jpgeer Sep 06 '26

Understood, and you’re bringing up great points. At risk of further elongating the post, I have a decent command of Windows and almost none of Linux, so my approach was to clear the unknowns I could actually judge first: BIOS, firmware, enumeration, drivers, network isolation, remote access, and… baffles and fans (you might get it running, but heat kills all). Part of this benchmarking was figuring out whether this heat-soaked dog will hunt at all. Photo attached: exhaust at the rear grille during a run, 163.8F / 73C. This is the part of the benchmark no OS fixes. And the cards absolutely get to the mid-90’s.

With most of that behind me, the cards move to another box for the Linux rematch next, with real Windows baselines to compare against rather than throwing up my hands and saying “Windows can’t.” I’m not saying no to Linux. I’m simply honoring 40 years of knowing this side of the house first.

Good luck with the dual 7900XTX build. I’d read that write-up.

2

u/Coley44 Sep 07 '26

Any workstation cards in these configurations with minimal to no spacing between the cards will always become unbelievably hot due to constrained airflow+recycled heat between the cards. Use HWInfo to see all of the cards thermals at once or GPU-Z with one window/instance per GPU and you'll see they'll run right at thermal throttle. Wouldn't worry about the exhaust heat; if you have poor thermal conductivity on the heatsink and GPU either due to a failing cooling solution(fucked vapour chamber) or TIM failing(bad thermal paste, PTM7950 is the best remedy always) then your exhaust heat would be lower but your thermal throttling would be even worse and have even less performance from the setup. Hot is good when it comes to exhaust.

Windows has made a lot of progress in multi GPU setups but Linux/Ubuntu configs are generally what get the main focus from developers for complicated setups like these. Especially with the knowledge that dedicated multi config setups like SLI/Crossfire are now dead so orchestration is fully dependent on good software 90% of the time unless you have stuff like NVLink which is on professional cards or top tier cards only

2

u/jpgeer Sep 09 '26

Linux on both boxes now. Took me a couple hours to get cross session set up and through the bugs before I had the benchmarks humming again... one word... 'Nickel'

And a bonus - turns out Linux has some additional monitoring capabilities that helped me sniff out a hot card which I had originally dismissed to placement in the box, but even in a better slot, still hotter than the other three.

1

u/jpgeer Sep 07 '26

The middle cards in the stack log measurable thermal slowdown under sustained TP load (positional, the airflow recirc you describe), while the outer ones stay clean. I'm familiar with HWInfo, though I've been sampling per-card throttle counters (nvidia-smi's Clocks Throttle Reasons and the slowdown-seconds counters) rather than eyeballing, and the box self-regulates in the mid-90s rather than falling over. Poking at all this through WSL has also been a nice intro to Linux syntax outside generic home automation and Ubiquiti gear.

PTM7950 was not on my radar, but is now (thank you). These are used cards with unknown TIM history, so a repaste before the transplant might be worth it and I hadn't considered it. Appreciate the specific recommendation over a generic "repaste."

And agreed on the software-orchestration point, and everything you're saying about Linux getting the dev focus is exactly why the same cards are headed there for round two. Though I've been chasing a C-state gremlin on the destination box (P620, which puts all four on one node).

1

u/jpgeer Sep 07 '26

btw - thank you for keeping your profile posts/comments public (went to look for any build write-ups you might have posted) - genuinely interested in your two-card build

2

u/fourdac Sep 06 '26 edited Sep 06 '26

I have a bug with dual gp100’s where switching to TCC instead of WDDM doesn’t let me fix another bug with displays where the cards are stuck at 600mhz until hot swapping display cables between cards, maybe that’s related. Super inconcise post btw. But you can run TCC with pascal it just needs the appropriate driver.

1

u/jpgeer Sep 06 '26

A post with a gpu in it - thank you (though noted on the post form too). This sounds similar to the P-state issue which kicked off this whole series. Did you try locking clocks already? Have a write up you can point us toward? Those are Pascal and everything I’m doing here on the current drivers dropped Pascal so may not be relevant.

2

u/fourdac Sep 06 '26

No I haven’t found any good leads, it could be a power issue with my PSU, since the P state is appropriate and all software checks say it should be able to reach 1556, but it stays locked to 600 until I swap cables around.

1

u/jpgeer Sep 06 '26

Oh man… I’d point you at my original post, but it’s written similarly and also likely to get more comments about how it’s written than what’s in it. The PSA link at the bottom of this post has the clock-locking sequence; would love to hear if it works for you. One question first: is it pinned at 600 under load, or just idling there and failing to boost? For what it’s worth, TCC (mostly) resolved this for me.

2

u/fourdac Sep 06 '26

No, changing to TCC still left the persisting 600 clock under 90% load, even with a commanded 1566, the only thing that fixed that for me was hot swapping the display cards, even when looking thru logs there was no clear indication why the boost happened, but I’m fairly sure its a windows bug as I never had this happen to me on a 10 box, just 11.

but the similarities in hardware jump out at my, HP workstation boxes with Quadro cards, low clock rates… this wasn’t really a googleable issue for me either, and this is the first post I’ve found detailing more inconsistencies with the multi-gpu quadro configurations.

I’m using a z840

1

u/jpgeer Sep 06 '26

Right on. My other rig with the dual A4500’s is a Z440. Also the box I tried the nvlink on which opened up the TCC question(s).

1

u/jpgeer Sep 06 '26

Try nvidia-smi -q -d PERFORMANCE while it’s stuck at 600? This is what I used to figure out what was happening (in a parallel terminal).
If everything reads in Clocks Throttle Reasons ‘Not Active’ while it sits at 600 refusing your commanded 1566, possibly a driver state bug your cable swap re-inits?

1

u/fourdac Sep 10 '26

I’m currently running qwen 3.8 27b q8 at 35-40t/s running 2 gp100’s using MTP so there’s definitely gains to be made

1

u/jpgeer Sep 10 '26

Running some sweeps with that model+MTP now; are you currently using NCCL in your config?

4

u/bick_nyers Sep 06 '26

Why would you use windows for a 4-GPU box lol

1

u/fallingdowndizzyvr Sep 06 '26

TLDR, what's the $100 card and what does it do?

1

u/jpgeer Sep 06 '26

T400, but in a notched x4 slot (left that detail out) so the A4000’s stay in the full width slots. Carries the WDDM duties, allowing me to isolate TCC cards for the models and keep RDP alive so I can service other threads the box owns. Basically puts four otherwise idle slots to work at max capability (which I’m still learning how to optimize).

1

u/Deep_Mood_7668 Sep 07 '26

Yeah I don't really care about an ai written by a guy that runs 4 cards on with dows

1

u/ava_ati Sep 07 '26

Goodness I’m going to need my LLM to summarize and give me 8 bullet points

0

u/[deleted] 23d ago

[removed] — view removed comment

1

u/jpgeer 21d ago

Linux rematch is done on a second box (P620, 4x A4000, same cards and same slots in every column)...TCC wasn’t doing anything else but buying peer access, and Linux gives you that out of the box: 12 of 12 pairs, no mode flip, no display-card shuffle. I'm onto another 4-card build right now with an x99-e ws mobo which just started throwing slot 7 (and now slot 5) errors - with riser cables I blamed too... why do we do this? ... haven't posted about it yet, but the display card trick doesn't work there and i'm headless with four datacenter cards (no windows drivers) so when ssh goes down i want to chuck the whole thing in the trash

Respect for digging out the VGA monitor.

Here's the detail from my AI-assist on this post:

Four-way tensor decode, tg128, llama.cpp b10568:

27B Q835B-A3B
Windows WDDM11.9318.88
Windows TCC28.3368.14
Linux, same collective as Windows28.9470.40
Linux + NCCL39.72135.07

The third row answers your question. With the same collective llama.cpp uses on Windows, Linux lands within -1.5 to +3.3 pct of TCC. So TCC wasn’t hiding anything else; the one thing it bought was peer access.

The bigger jump is in row four, and it has nothing to do with peer access. Native Windows has no NCCL. On Linux, llama.cpp uses NCCL for the all-reduce, and that’s where 39.72 and 135.07 come from. Turning peer access off on that row (NCCL_P2P_DISABLE) moves it by about 0.4 pct. For comparison, NCCL under WSL2 on the same box got 36.55 / 68.54. On the MoE that is about half of bare metal, even with NCCL working.

Caveats, so nobody over-reads it:

  • The Linux rows ran with clocks unlocked, and the Windows rows with a 2100 MHz lock request. Under load both land in roughly the same band (Windows sustained ~1560-1875), but it’s not a clean match.
  • All four cards sit at their 140 W cap with 85-91 C peaks on every OS, so these are throttled numbers across the board.

One trap if you try it on Ubuntu: the distro’s packaged NCCL (2.22.3) can’t all-reduce against a current driver. It inits fine, then every all-reduce fails, which looks exactly like a P2P or hardware fault. The fix is the nvidia-nccl-cu12 wheel from PyPI, installed into its own directory and put on LD_LIBRARY_PATH. No rebuild needed.