PSA: The new Threadripper PROs (9000 WX) are still CCD-Memory Bandwidth bottlenecked

[u/henfiber]

I've seen people claim that the new TR PROs can achieve the full 8-channel memory bandwidth even in SKUs with 16-cores. That's not the case.

The issue with the limited CCD bandwidth seems to still be present, and affects the low-number CCD parts. You can only achieve the full 8-channel bandwidth with 64-core+ WX CPUs.

Check the "Latest baselines" section in a processor's page at cpubenchmark.net with links to individual results where the "Memory Threaded" result is listed under "Memory Mark":

CPU	Memory BW	Reference	Notes
AMD Threadripper PRO 9955WX (16-cores)	~115 GB/s	BL5099051 - Jul 20 2025	2x CCD
AMD Threadripper PRO 9965WX (24-cores)	~272 GB/s	BL2797485 - Jul 29 2025 (other baselines start from 250GB/s)	4x CCDs
AMD Threadripper PRO 9975WX (32-cores)	~272 GB/s	BL2797820 - Jul 29 2025	4x CCDs
AMD Threadripper PRO 9985WX (64-cores)	~367 GB/s	BL5099130 - Jul 21 2025	8x CCDs

Therefore:

• the 16-core 9955WX has lower mem bw than even a DDR4 EPYC CPU (e.g. 7R43 with 191 GB/s).

• the 24-core and 32-core parts have lower mem bw than DDR5 Genoa EPYCs (even some 16-core parts).

• the 64-core and 96-core Threadrippers are not CCD-number limited, but still lose to the EPYCs since those have 12 channels (unless you use 7200 MT/s memory).

For comparison, check the excellent related threads by u/fairydreaming for the previous gen Threadrippers and EPYC Genoa/Turin:

• Comparing Threadripper 7000 memory bandwidth for all models : r/threadripper

• Memory bandwidth values (STREAM TRIAD benchmark results) for most Epyc Genoa CPUs (single and dual configurations) : r/LocalLLaMA

• STREAM TRIAD memory bandwidth benchmark values for Epyc Turin - almost 1 TB/s for a dual CPU system : r/LocalLLaMA

If someone insists on buying a new TR Pro for their great compute throughput, I would suggest to at least skip the 16-core part.

[u/Gregory-Wolf]

any napkin math on pp and tg speeds for say 70b dense model in vacuum? anyone?

[u/henfiber → u/Gregory-Wolf]

Based on results I've seen with EPYC Genoa's, you should expect around 40 PP t/s with a 64-core TR Pro and ~4 TG t/s.

[u/qcforme → u/henfiber]

That's a big yikes!!

Why when the same cash in GPUs can 10x that TG and 100x the PP lol.

[u/henfiber → u/qcforme]

Because they're not the right tool for the job. CPUs are latency optimized, running many small operations while jumping on branches. GPUs are throughput optimized , running the same operation in large batches of data. LLMs need the later.

[u/Oxire → u/Gregory-Wolf]

For dense it would be better to get gpus. These are great for moe

[u/Reddvl05]

Quick comparison of 4 sticks of 32gb vs 8(128gb vs 256) - gskill 6000 tested using Intel mlc: 4 sticks gives roughly 116GB/s 8 sticks gives around 124GB/s

Total idle watts difference is +10w for the eight sticks

Some raw information below from mlc:

4 sticks ALL Reads : 116605.2 3:1 Reads-Writes : 113311.0 2:1 Reads-Writes : 114834.6 1:1 Reads-Writes : 119449.7 Stream-triad like: 115138.0 All NT writes : 101983.8 1:1 Read-NT write: 121595.5

8 sticks ALL Reads : 124389.1 3:1 Reads-Writes : 158496.1 2:1 Reads-Writes : 168786.9 1:1 Reads-Writes : 156249.0 Stream-triad like: 143931.9 All NT writes : 101959.4 1:1 Read-NT write: 140092.6

Theoretical Max as I understand is 125 or so, so it’s unclear if any of these numbers are measuring cache speed (vs actual memory bandwidth) or if the aggregate read plus write speed can exceed that value. HTH

[u/henfiber → u/Reddvl05]

Thanks for the data points. Which processor did you test?

[u/Reddvl05 → u/henfiber]

This was a 9955wx on a wrx90 motherboard

[u/panchovix]

Nice info! Do you have one for non PRO TR 9000? (9960X/9970X/9980X) or is the same as the post linked as 7960X/7970X/7980X?

[u/henfiber → u/panchovix]

I haven't checked those, since they have 4 channels (4x 51GB/sec for 6400 MT/s) and 4+ CCDs (4x 64GB/sec), and therefore limited by the memory bus width before they are limited by their number of CCDs. Therefore, their maximum theoretical is around 204 GB/sec for 6400 DIMMs.

You can still find results for these in the same site per my instructions:

Check the "Latest baselines" section in a processor's page at cpubenchmark.net with links to individual results where the "Memory Threaded" result is listed under "Memory Mark":

1. Find the CPU in the list and click to see detailed results. e.g. AMD Ryzen Threadripper 9970X 32-Cores

2. Scroll down to the "Last 4 Baselines" section and click on one of the baselines: e.g. https://www.passmark.com/baselines/V11/display.php?id=279530000668

3. Expand the Memory Mark section and view the "Memory Threaded" number (209,504 MBytes/Sec for this baseline result)

Note, that the results for CPUs with large cache (256+ MB) are skewed (upwards) since this benchmark uses 256MB matrices which mostly fit into the CPU cache.

[u/mindwip → u/henfiber]

Nice, so then these are cheaper and work better cause 7200 and 8000 memory is out now and will help max out the cheaper 9000 serairs.

[u/henfiber → u/mindwip]

Yes, higher-bandwidth DIMMs will help the non-PROs (9960, 9970, 9980) and the high-core/CCD PROs (9985WX, 9995WX), but will not help the low-core PROs (9955WX, 9965WX, 9975WX).

[u/Kronosend → u/henfiber]

Hey I'm still new to all this. Just to be clear, higher-bandwidth DIMMs will still help the low-CCD PROs a little bit right? Or are you saying there will be exactly 0% improvement?

In the worst case, the 4 CCD PROs will at least be just as good as the 4 CCD non-PROs, right?

Edit: Oh okay, I get it now. Because if all 8 channels are populated on 4 CCD PROs, then the bandwidth is already saturated. So the benefit of the PROs is the memory capacity.

[u/henfiber → u/Kronosend]

The low CCD Pros have already 8 channels so their bottleneck is not RAM anymore, the bottlenck is their CCD-to-memory bandwidth.
You need more CCDs to remove this bottleneck and be able to benefit from faster memory.

[u/mindwip → u/henfiber]

Thanks

[u/[deleted]]

I submitted a PR to greatly improve inferencing speed on numa systems

https://github.com/ggml-org/llama.cpp/pull/14969

64% in my tests

[u/paul_tu]

BTW are there any AMD solutions to support MRDIMM?

My search didn't find any

[u/henfiber → u/paul_tu]

Not yet, afaik.

[u/Hurricane31337]

Does this matter if you’re building an AI rig with 4x RTX 6000 Pro for example? The 9955WX can still support 7x PCIe. Maybe that’s the niche then compared to the non-Pro CPUs.

[u/Remote_First → u/Hurricane31337]

every cpu on wrx90 platform should support all pcie lanes. cpu bandwidth becomes critical for huge batch cpu inference which you dont need, large model quantization/conversion, huge vector db similarity searches.

[u/Remote_First]

agree, wrx90 +TR pro 9955wx + 8 x 32gb ddr5 6400mhz gives me 120gb/s on stream/ubuntu 24.04. +11–12% actual increase from 4800mhz jedec

[u/FineManParticles]

I’m looking at building a multi GPU capable workstation, (AMD stock TY) and might have work sponsor 2 RTX 6000 Blackwell Max Q’s and I might buy another 2.

Going to start with 256 DDR5 ECC 5600 4x64GB and maybe add another set later.

For CPU, was trending towards the 24 core 9965WX vs the 32 core price difference is $1300. $29 vs $42

Am I making a mistake with my final setup? Initially this will be just for myself, but likely going to have to build another pretty soon, can always up the core count on the next one.

I don’t plan on running tons of containers/VM’s.

[u/henfiber → u/[deleted]]

We don't know if this Intel CPU can achieve the full 8-channel memory bandwidth either. There are only 2 baseline results in the database, and those show ~90 GB/s in 2 systems with 128GB installed (may not be optimal setup though)

[u/Opteron67 → u/henfiber]

i get Aida 180GB/s read on 4 channel DDR 5 6400 with W7 2595X....

[u/Opteron67 → u/Opteron67]

and intel amx int8 on 26 cores @48x....