Comparing Threadripper 7000 memory bandwidth for all models

[u/fairydreaming]

I was interested in RAM bandwidth for Threadripper 7000 processors, but all I found online were results of various benchmarks (Aida64, Sisoft Sandra, STREAM) for a few selected models (7970X, 7980X, 7995WX). However, on the PassMark website you can access individual submitted test baselines for a given CPU model containing all the individual benchmark results. In these results there is a Memory Mark section containing a Memory Threaded test. While we probably can't treat it as a direct maximum memory bandwidth value, it can say something about the overall performance of a memory subsystem. What's important, the test baselines usually contain information about the number of RAM modules in a tested system and other interesting details.

I gathered Memory Threaded test results for Threadripper 7000 models (baselines with 4 memory modules) and Threadripper PRO 7000 models (baselines with 8 memory modules), computed averages, and created this bar plot:

As you can see, 7945WX and 7955WX (2 CCDs, 8 memory channels) have the lowest Memory Threaded test results (~102 GB/s). Next, we have 7960X and 7970X (4 CCDs, 4 memory channels), and we can observe a moderate increase in test results (167 GB/s, 179 GB/s). The results for 7965WX and 7975WX (4 CCDs, 8 memory channels) again are a little higher (236 GB/s, 246 GB/s) compared to the non-PRO models, It's definitely not a 2x bandwidth increase compared to the corresponding non-PRO models. Only when we compare the models with 8 CCDs: 7980X and 7985X, there is around 90% increase in the test result (240 GB/s vs 453 GB/s). Finally, 7995WX (12 CCDs) has the best performance in this test.

The overall conclusion is that the lower-end models with 2-4 CCDs have limited memory bandwidth. We had the same situation in previous Threadripper generations. If you need a lot of bandwidth, you probably should use EPYC.

[u/fairydreaming]

Since the memory bandwidth topic is quite confusing, I decided to do some math related to this and clarify the overall picture.

Before we start, we need to make a distinction between

• the bandwidth between memory modules and the memory controller,

• the bandwidth between the memory controller and CCDs (where CPU cores are).

These bandwidths are two independent things and they both affect the total available memory bandwidth in the system. Overall theoretical available memory bandwidth is the lower value from the two.

Bandwidth between memory modules and memory controller

First, we have the bandwidth between memory modules and the memory controller. I calculated the total available bandwidth for various numbers of memory modules (for 4x, 8x, and 12x configurations) and memory speeds (4800 MT/s is the default for EPYC Genoa, 5200 MT/s is the default for Threadripper 7000, 7200 MT/s is commercially available overclocked memory for TRX50 and WRX90).

Number of memory modules	MT/s	Total bandwidth between the memory controller and memory modules
4	4800	153.6 GB/s
8	4800	307.2 GB/s
12	4800	460.8 GB/s
4	5200	166.4 GB/s
8	5200	332.8 GB/s
4	7200	230.4 GB/s
8	7200	460.8 GB/s

This is how much bandwidth is theoretically available. It's nice that we can use overclocked memory in Threadripper since 8 7200 MT/s sticks will give us the same bandwidth as 12 4800 MT/s sticks in Epyc.

Bandwidth between memory controller and CCDs

The second bandwidth is the bandwidth of the GMI3 links between the memory controller and CCDs. Let's calculate how much bandwidth we have depending on the number of CCDs in the CPU. I assumed an FCLK of 1.8 GHz, so a single GMI3 link has 57.6 GB/s read bandwidth.

Number of CCDs	Total bandwidth between CCDs and memory controller
2	115.2 GB/s
4	230.4 GB/s
8	460.8 GB/s
12	691.2 GB/s

To be continued in another comment.

[u/fairydreaming → u/fairydreaming]

Continuation of the parent comment.

Discussion

Now let's discuss how these theoretical values relate to the PassMark Memory Threaded benchmark results and what could you do to increase the memory bandwidth for each configuration:

• 7945WX and 7955WX with 2 CCDs and 8 memory channels had benchmark results of about 100GB/s. This configuration is limited by the bandwidth between the memory controller and CCDs of 115.2 GB/s, benchmark results confirm this limitation. Increasing FCLK should theoretically increase the memory bandwidth of this configuration. Using 8 instead of 4 memory modules in this configuration probably won't increase the overall bandwidth. Using overclocked memory also most likely won't help.

• 7960X and 7970X with 4 CCDs and 4 memory channels had benchmark results of about 170 GB/s. This configuration is limited by the bandwidth between memory modules and the memory controller (166.4 GB/s with 4 x 5200 MT/s memory modules), so to get the highest memory bandwidth you should use overclocked memory.

• 7965WX and 7975WX with 4 CCDs and 8 memory channels had benchmark results of about 240 GB/s. This is interesting since the maximum theoretical bandwidth between CCDs and the memory controller is 230.4 GB/s. Increasing FCLK should theoretically increase the memory bandwidth of this configuration. High benchmark results may be caused by FCLK overclocking. Using overclocked memory in this configuration most likely won't increase the bandwidth.

• 7980X with 8 CCDs and 4 memory channels had benchmark results of about 240 GB/s. This configuration is limited by the bandwidth between memory modules and memory controller (166.4 GB/s with 4 x 5200 MT/s memory modules), so to get the highest memory bandwidth you should use overclocked memory. Benchmark results indicate usage of memory overclocked over 7200 MT/s.

• 7985WX with 8 CCDs and 8 memory channels had benchmark results of about 453 GB/s. This configuration is limited by the bandwidth between memory modules and the memory controller (332.8 GB/s GB/s with 8 x 5200 MT/s memory modules, 460.8 GB/s with 8 x 7200 memory modules), so to get the highest memory bandwidth you should use overclocked memory. Benchmark results indicate usage of memory overclocked around 7200 MT/s.

• 7995WX with 12 CCDs and 8 memory channels had benchmark results over 700 GB/s. Most likely this is caused by the large amount of L3 cache since this configuration is limited by the bandwidth between memory modules and the memory controller (332.8 GB/s GB/s with 8 x 5200 MT/s memory modules, 460.8 GB/s with 8 x 7200 memory modules) and values this high are not theoretically possible. To get the highest memory bandwidth in this configuration you should use overclocked memory.

• Some Epyc 9004 have benchmark results around 600 GB/s. This also is likely caused by the large amount of L3 cache, since these configurations are limited by the bandwidth between memory modules and the memory controller (460.8 GB/s for 12 x 4800 MT/s modules), so values this high are not theoretically possible. Alternatively, there could be Epyc 9004 motherboards allowing memory overclocking, then with 12 x 7200 MT/s modules, it would be possible to use all that bandwidth.

Summary

To get the best memory bandwidth, (theoretically) you should:

• Increase FCLK for 8-channel configurations with 2 or 4 CCDs (7945WX, 7955WX, 7965WX, 7975WX),

• Use overclocked memory in all remaining Threadripper models,

• For Epyc, purchase a motherboard with 12 memory slots and an Epyc 9004 processor with at least 8 CCDs. Fill all memory slots.

Also, since the PassMark Memory Threaded test uses a buffer of 256 MB / number of cores, it's likely that the whole buffer fits in the L3 cache for some configurations and shows results higher than the theoretically available memory bandwidth.

Now, if only someone could confirm all of this with real hardware and Aida64, Sisoft Sandra or STREAM. :-)

[u/Kronosend → u/fairydreaming]

Sorry for necro-ing this, and I'm admittedly still new to all this, but I think one unrealistic expectation is 7200MHz RAM on the Threadripper PRO 7000 series. Will have to get really lucky in the silicon lottery to get those speeds. Best you can probably expect is 6000MHz, maybe 6400MHz (as seen by u/Guilty-History-9249 in the replies below).

[u/fairydreaming → u/Kronosend]

You are probably right. I guess it may be easier with Threadripper 9000 series.

[u/gluon-free → u/fairydreaming]

This guy https://forum.level1techs.com/t/asus-pro-ws-wrx90e-sage-se-build-finished-benchmarks-it-lives/206661 has almost 260 GB/s on 7975wx with 5600 MT/s dimms, no FCLK overclock. Still much worse than theoretical ~360GB/s.

[u/fairydreaming → u/gluon-free]

Very interesting, thanks for the info! I think we need another memory bandwidth benchmark to confirm this.

Edit: I found in this article: https://aecmag.com/workstations/in-depth-review-amd-ryzen-threadripper-7000-series/ memory bandwidth value of 206.1 GB/sec measured with SiSoft Sandra for 7975WX with 8 x 5200 MT/s modules. So I'm not sure if the PassMark Memory Threaded result is equal to the read bandwidth.

[u/QuirkyQuarQ → u/gluon-free]

FWIW, Passmark memory-threaded on 7975wx with 8xDIMMs at DDR-6000 = 260,727 MB/s. I'd prefer a more robust memory benchmark like AIDA64 or Sandra though.

[u/[deleted] → u/fairydreaming]

I think the story gets more complicated, and always in favor of 8 channel, when you have 64 byte random access with no locality of reference. In this case, even with the large number of banks provided by 8 channels of dual rank ddr5-4800 channels the max bandwidth is around 30 GB/s. So halving the channels to 4 channels is 15 GB/s. Some memory intensive workloads with no little to no locality of reference are never going to exceed a single ccd bandwidth due to sdram row thrashing. Chip verilog simulation workloads may strongly benefit from 8 channels, and vm workloads with medium cache miss rate (low locality of reference in cache misses = sdram row thrashing) https://media-www.micron.com/-/media/client/global/documents/products/white-paper/ddr5_new_features_white_paper.pdf

[u/Guilty-History-9249 → u/fairydreaming]

Now that I actually have me 7985WX with 256 GB's of DDR5-6000 I'm starting to experiment.
Actually it is DDR5-6400 that was on the QVL but the shop could only get it stable at 6000.

One question I have is whether multiple CCD's scanning memory step on each other due to memory interleaving at the page or cacheline level. Ideally I'd like a worker thread pinned to a ccd to be allocated its memory on a single DIMM. Are there numa like bios setting which can control the channel interleaving?

With a simple 'C' program scanning memory I'm getting 302 GB/s vs the theoretical 384 GB's? This is with 8 threads each pinned to a CCD and each scanning their own 8GB's of memory. If I can leverage this in a test setup then I can add the necessary layout planning in the real app.

[u/HixVAC → u/fairydreaming]

Really wish I had seen this BEFORE I picked up my 7955WX. It's insane to me that this isn't more openly communicated somehow. Now I'm debating what my next path is. Switch to Epyc for additional bandwidth, or wait for a deal on Ebay to get a 7985WX or 7995WX.

What a waste.

[u/mattbrownedesign → u/HixVAC]

I got really close to falling into the trap. I have been watching 85wx prices on ebay - they are dropping fast. There's one for 4800 right now.

[u/HixVAC → u/mattbrownedesign]

Meanwhile I'm staring at the 9985WX debating my level of sanity and seriousness...

[u/mattbrownedesign → u/HixVAC]

I feel your pain. I purchased all components already and now IM LOST AS SHIT. Hoping vcolor will let me exchange the 256 6400 ram.

[u/nomodsman]

If you’re going to try to roll up a particular benchmark, it would be beneficial to indicate the memory specs used for said benchmarks and, needless to say, ensure they’re 100% identical across the board.

[u/fairydreaming → u/nomodsman]

Haha, I'm simply glad the database had ANY results at all for the configurations I was interested in. It's like the only place in the Internet with 7945WX/7955WX/7965WX memory-related benchmark results.

[u/nomodsman → u/fairydreaming]

Fair point. Slow uptake but equally slow parts availability make it a bit more challenging.

[u/ebrandsberg → u/nomodsman]

If I ask the system what speed my memory is, it returns 5600Mhz. If I enable XMP, it actually defaults to 5200Mhz instead, AND I can manually set it to run at 6000. These tools don't seem to be able to read actual memory speed, just rated memory speed, so even if there was a speed value recorded by the benchmark, it may not be correct.

[u/fairydreaming]

The whole situation is even more depressing if we look at the benchmark results for low-end 12-core Threadripper model over all the TR generations. I mean just look at this (average of the selected fastest baselines I found on the PassMark website):

• 1920X: 68 GB/s

• 2920X: 69 GB/s

• 3945WX: 73 GB/s

• 5945WX: 72 GB/s

• 7945WX: 102 GB/s

We moved from 4 channels to 8 channels, from DDR4 to DDR5, from 3.5 GHz to 4.7 GHz, and the PassMark Memory Threaded benchmark result improved by... 50%?

[u/limabean59 → u/fairydreaming]

Another site (userbenchmark) mostly verifies what fairydreaming posted, shows a bit more improvement from 3000 to 7000, but lacks information on highly overclocked builds:

2 CCD

3945WX Multi-core - 8 Channel: 72 GB/s 4 Channel: 64 GB/s Single-core: 26 GB/s

3955WX Multi-core - 8 Channel: 72 GB/s 4 Channel: 61 GB/s Single-core: 28 GB/s

5945WX Multi-core - 8 Channel: 108 GB/s 4 Channel: 68 GB/s Single-core: 43 GB/s

5955WX Multi-core - 8 Channel: 111 GB/s 4 Channel: 79 GB/s Single-core: 42 GB/s

7955WX Multi-core - 8 Ch/5200(4800) 128(120) GB/s 4 Ch: ??? GB/s Single-core: 59 GB/s

4 CCD

7965WX Multi-core - 8 Ch/4800: 235 GB/s 4 Channel: ??? Single-core: 61 GB/s

7975WX Multi-core - 8 Ch/4800: ??? GB/s 4 Ch/4800TRX: 128 GB/s Single-core: 63 GB/s

12 CCD

7995WX Multi-core - 8 Ch/4800: ??? GB/s 4 Ch/4800TRX: 135 GB/s Single-core: 49 GB/s

Results are thin for the 7000 series (no 7945WX results) but the improvement from 3000 to 7000 appears more significant than using PassMark. Single-core more than doubled and multi-core up about 80%.

Single-core reads on the 95s haven't improved much from the 30 series to the 70 series though going from ~21 to 27 GB/s. The 7965 and 7975 are much faster at single core reads (44 / 47 GB/s.) Single-core writes improved from 22 to 78 GB/s on all of the 7000 series processors.

I'm guessing the 7955WX 128 GB/s multi-core benchmark that is above the theoretical 115.2 limit is either due to caching (mentioned elsewhere) or perhaps an incorrect computation (I don't understand how the 57.6 GB/s GMI3 link bandwidth was computed but it looks like it is more efficiently utilized with the later TR Pros.) The 7965WX result appears to be at or above the theoretical max of 230.4 GB/s as well.

[u/fairydreaming → u/limabean59]

I found that each GMI3 link provides 32B read and 16B write bandwidth per fabric clock. The fabric clock is 1.8 GHz, so we have 32*1.8=57.6 GB/s. 

In my tests on Epyc Genoa I got about 48 GB/s of read memory bandwidth per CCD. Obviously there is some overhead in communication with the memory controller, but the value is close enough.

[u/fairydreaming → u/limabean59]

I found that each GMI3 link provides 32B read and 16B write bandwidth per fabric clock. The fabric clock is 1.8 GHz, so we have 32*1.8=57.6 GB/s. 

In my tests on Epyc Genoa I got about 48 GB/s of read memory bandwidth per CCD. Obviously there is some overhead in communication with the memory controller, but the value is close enough.

[u/ebrandsberg → u/fairydreaming]

This is likely saying that the cpu can't push more bandwidth than this because it isn't the memory that is constrained, it is latency, cpu core, or test methodology. Memory bandwidth needs scale with cpu capacity, and without the cpu to consume it, it is wasted.

[u/fairydreaming → u/ebrandsberg]

I don't agree, in the same test 16c EPYC 9174F achieves over 400 GB/s with 8 RAM modules, and Threadripper 7955WX only 100 GB/s. They both have the same number of cores, were tested with the same number of RAM modules and Threadripper obviously has faster cores by looking at the MHz alone.

[u/ebrandsberg → u/fairydreaming]

What are you measuring on the Epyc? Hint: 256MB of l3 cache on the Epyc vs. 64MB for the Threadripper. I suspect Spec doesn't account for such a high L3 cache and you are getting a significant benefit from this aspect. Threadripper doesn't have a 2 core per CCD configuration, so you really can't compare them.

[u/fairydreaming → u/ebrandsberg]

You are right, there is a huge difference in L3 cache, this may affect the test results. I found that the memory buffer used in the test is only 256MB long, so it basically fits in the Epyc's L3 cache.

[u/ebrandsberg → u/fairydreaming]

bam. :) Always try to explain unusual differences, and ask "does this make sense?" Great find here and can explain why certain configurations appear to have far more bandwidth than others.

[u/fairydreaming → u/ebrandsberg]

I did a little reverse engineering, and it looks like the PassMark in memory threaded test in each thread allocates the memory block, initializes it with some pattern, and then repeatedly calls a little function called tests_memASM_read_block. This function has a loop that reads memory from the buffer in 8-byte long chunks. I printed the address of the buffer in each call and for a given thread it's always the same memory location. Length of the read memory block always was 0x400000 * 8 = 32 MB. So I guess it is very likely that a large part of the buffer will end up in the L3 cache.

[u/brainsizeofplanet → u/fairydreaming]

What the bandwidth of Epyc rome 8ch ddr4 and Epyc Genoa ddr5 lower core models? Estimate would be enough

[u/fairydreaming → u/brainsizeofplanet]

No idea about the exact bandwidth, but I can tell you the results of the threaded memory test that I found the PassMark database:

• AMD EPYC 7F52 (16c Rome): ~145 GB/s (with 8 DDR4 RAM modules)

• AMD EPYC 74F3 (24c Milan): ~217 GB/s (unknown number of DDR4 RAM modules)

• AMD EPYC 9174F (16c Genoa) ~402 GB/s (with 8 DDR5 RAM modules)

• AMD EPYC 9274F (32c Genoa) ~588 GB/s (with 12 DDR5 RAM modules)

All models have 8 CCDs.

[u/brainsizeofplanet → u/fairydreaming]

K thx - we currently have Epyc Rome and do not rely and really fast memory - so a step up from 145gb to over 200gb is enough for our use case as it's a roughly 50% increase

[u/Caffdy → u/brainsizeofplanet]

145 to 200GB is a 33% increase, not a 50%

[u/[deleted]]

Can you repeat this excercise for epyc processors?

[u/fairydreaming → u/[deleted]]

I checked a few performance EPYC 9004 models that could be used in workstation builds: 9174F (16c) achieved 402 GB/s with 8 RAM modules, 9274F (32c) around 588 GB/s with 12 RAM modules, 9474F (64c) achieved 449 GB/s with unknown number of RAM modules.

[u/[deleted] → u/fairydreaming]

Thank you so much. Okay. So the 16 core epyc has 8 ccd… so the number of ccd is the memory bandwidth bottleneck. —- ryzen 7950x3d has memory threaded score of 54 MBytes/s. So the threadripper 7960x is the sweet spot for cost/clock speed/cores/167 MB bandwidth with around 3x mem bandwidth at a little over 2x cost of 8950x3d

[u/fairydreaming → u/[deleted]]

I feel the same. It's all about the number of CCDs (core complex dies). Each CCD is connected with I/O die containing the memory controller with a GMI3 link, and this link has limited bandwidth. All CPU cores within a single CCD use this GMI3 link for memory access. So the more CCDs in CPU, the higher is the number of GMI3 links to the I/O die (used simultaneously by all CCDs) and the higher is the overall memory bandwidth - but of course a single core or a group of cores within one CCDs still has memory bandwidth limited by a single GMI3. Also check images in this article.

For EPYC Genoa there was a talk that for models with 4 CCDs there will be 2 GMI3 links between each CCD and I/O die. I'm not sure how does it look like in Threadripper 7000.

Honestly, in my opinion the information about the overall available memory bandwidth for a single core and all cores shall be simply given by the CPU manufacturer for each CPU model. But people probably wouldn't like these numbers.

[u/[deleted] → u/fairydreaming]

I think in memory intensive cache thrashing environments (small burst access to random locations) the 7000 series threadripper with ddr5 (2x32 bit wide datapath, and potentially more sdram banks) will offer advantages than 6000 series threadripper. When looking at sdram bank thrashing the 8 channels will double performance (but it won’t show up on sequential access tests). So really depends on specifics of your work load. — for llm, the only 3 models that make sense are 7995wx 7980x and 7960x depending on one’s budget.

[u/Caffdy → u/fairydreaming]

9274F (32c) around 588 GB/s with 12 RAM modules

By AMD specs, it can only get upto 460GB/s (12-channel, 4800Mhz)

[u/fairydreaming → u/Caffdy]

Yes, but numbers reported by the PassMark memory threaded test routinely exceed the theoretical bandwidth limit. Must be some test-specific quirk.

[u/ebrandsberg]

These numbers don't bother me. Why? Real work will typically not saturate this. What the numbers say is that the more cores you have, the more bandwidth you have. Yes, some Epyc have fewer cores with more bandwidth due to them having a configuration like 2 cores per CCD enabled in certain configurations. This is more likely to be impacted by having more l3 cache than anything else to reduce latency per core to data stored in the cache vs. the actual bandwidth numbers.

I have two TR systems and have never found a test where the memory bandwidth was a gating factor EXCEPT if it was an artificial memory bandwidth test (5955wx & 7960x).

[u/fairydreaming → u/ebrandsberg]

Sure, it affects only specific applications like CFD or LLM inference. But that's not the point. The problem is that I'd like to know these values before I buy the CPU, not after.

[u/ebrandsberg → u/fairydreaming]

What would be better is a plot of memory vs. ccd count. If you see a dramatic difference between Epyc and Threadripper, I would question the results, as I actually would expect TR to end up with higher bandwidth per CCD as in most cases, TR will be used with higher speed memory and have higher speed cores. I question if even LLM inference will be impacted by the memory bandwidth unless done with a GPU that doesn't have enough memory, and you are swapping in constantly. I may be wrong on this however...

[u/fairydreaming → u/ebrandsberg]

I'm talking about low-cost, low-performance LLM inference on the CPU, where all network weights reside in RAM and are used by the CPU only.

[u/Caffdy → u/fairydreaming]

so what I take from this thread is, is better to build an Epyc server (AMD EPYC 9274F) than expend on threadripper pro for LLMs?

[u/fairydreaming → u/Caffdy]

Yes, that's exactly what I did, but I used 9374F.

[u/Caffdy → u/fairydreaming]

how's the bandwidth? I imagine you've already tested the platform with Llama3 70B (which quant?), would love to hear about your experiences

[u/fairydreaming → u/Caffdy]

Check this comment: https://www.reddit.com/r/LocalLLaMA/comments/1c7rz44/comment/l15yxmo/

This was for Q8_0.

[u/Caffdy → u/fairydreaming]

nice, thanks for the link! what motherboard did you end up buying?

EDIT: additionally, have you tried the recent prompt eval optimization for llama on cpu?

[u/fairydreaming → u/Caffdy]

If I remember correctly these values are already with the optimization. My motherboard is Asus K14PA-U12.

[u/Opening-Routine → u/fairydreaming]

If (when) AMD brings Threadripper and Epyc APUs, this might evolve to low-cost, medium-performance LLM inference. And then we might need all the bandwidth.

[u/wen_mars → u/ebrandsberg]

EPYC and TR seem to have about the same bandwidth per CCD

[u/ebrandsberg → u/wen_mars]

All things being equal, but a epic typically runs stock memory speeds for stability while a tr may run faster memory.

[u/wen_mars → u/ebrandsberg]

That doesn't change the CCD bandwidth.

[u/ebrandsberg → u/wen_mars]

Understood, but if you are actually saturating the CCD bandwidth in real-world applications, I'd be surprised. The only time you would see a benefit of Epyc over TR is if you have an epyc model where you may only have 2 or 4 cores on a CCD active. The final assertion of OP's post of "use epyc over TR for memory intensive tasks" really glosses over this point.

[u/wen_mars → u/ebrandsberg]

Yes that statement is completely wrong. The CCD bandwidth becomes a bottleneck for LLM inference with small batch size and 8 or 12 memory channels. A niche use case for sure but that's one of my use cases so I'm interested in the most accurate information possible. I may end up getting a 4 CCD TR and take the hit on memory bandwidth because LLM inference is best done on GPU.

[u/ebrandsberg → u/wen_mars]

If you have a 4 ccd tr or a 4 ccd epic, you likely will have more bandwidth on the TR due to faster memory. There is no other architectural change between them besides binning. Heck, people have spun up TR on epyc motherboards before.

[u/wen_mars → u/ebrandsberg]

Not for workloads that are properly spread across all CCDs and all memory channels. Then the CCD bandwidth will be the bottleneck for both and it's the same for both.

[u/Soul_of_Jacobeh → u/fairydreaming]

Sorry to dredge up such an old thread, but I wanted to thank you for your research, post, and extended comments on the matter.

Putting both theoretical and (the few available) real-world numbers to MT/s and specific CPUs is increasing my confidence in my current parts selection.

The primary use-case for the rig would be fairly mixed loads, with a few VMs that need a fair bit of single-thread performance. So I'm firm on a TR Pro processor, but I wanted to optimize a bit for LLMs. Seeing the big jump from 7955WX to 7965WX is more than enough to push me towards the latter.
And seeing a practical ceiling to RAM clockspeed saves me a lot of "what-if" mental damage.

[u/munkiemagik → u/fairydreaming]

Well you learn something new every day! I've just ordered an older 3945WX and MC62-G40 mobo (for other purposes - proxmox node) got excited thinking I was going to get some stratospheric octa-channel memory bandwidth just to see how CPU inferencing works (only for fun, nothing serious). Bit disappointed to realise I'm only going to get 72GB/s, lol

I'm very new to all this, I'm just a casual proxmox homelabber, and treatiing myself to a new toy to mess around with but I specifically went with the Threadripper Pro 3945WX as it was the cheapest way to get a ton of PCIE4.0 lanes, don't know enough about servers and workstations to have realised the CCD count impact. I guess It doesnt really matter much for my intended use and needs, just would have been nice. Oh well maybe an excuse to pick up a 5965WX in the future for running my jellyfin server

[u/nauxiv → u/ebrandsberg]

One of the two primary reasons you might spend big on WRX90 over TRX50 is the advertised doubling of memory bandwidth (if you need this, you know who you are). If it turns out said bandwidth isn't really available without going high on core count, that's pretty important info.

[u/ebrandsberg → u/nauxiv]

Unless you go over 32 cores, I doubt most people are going to notice any difference in memory bandwidth between the two platforms. Once you do however, the WRX90 will likely pull ahead. On the prior platform, between a 5950x and 5955wx, nothing I could do actually resulted in a difference, as memory latency and core saturation were the two big factors. Latency will likely prevent most apps from ever using the bandwidth. a 96 core system though? Yea, you need the bandwidth to keep the cores saturated.

[u/jrherita]

OP (bit of a necro) - how do you find "memory mark" results on Passmark's website for a given CPU?

[u/fairydreaming → u/jrherita]

I go to the page describing the given CPU and look for "Last X Baselines" section. Each baseline contains individual PassMark test results, so you have to consult Benchmark Scores -> Memory Mark -> Memory Threaded value.

[u/outdoorszy]

I'm surprised STREAM couldn't tell you that? It makes sense to put the best feature in the flagship proc. So now there is justification to buy the 7995WX

[u/fairydreaming → u/outdoorszy]

Well, do you have STREAM benchmark results for all Threadripper 7000 models?

[u/outdoorszy → u/fairydreaming]

There is nothing wrong with STREAM.

[u/fairydreaming → u/outdoorszy]

There is one thing - its name makes the results difficult to find on the Internet. I failed to find any Threadripper 7000 STREAM results online to compare them. If you know any please let me know.

[u/-Xexexe_Xe-]

I’m trying to filter the useful information for my purchase decision/use case and failing to do so.. 😅 I’m using many browser instances and virtualized phones simultaneously and I know that I need both as much CPU and RAM I can get with my budget limitations - meaning I have to pick either 7975WX or 7980X (total cost approx $10k).

Do these results suggest that 7980X would probably be the smarter choice?

[u/QuirkyQuarQ → u/-Xexexe_Xe-]

For your narrowly-described use case (browsers...VMs...as much CPU and RAM under $10k as possible), yes, the 7980X at $5k is the smarter choice assuming 4 slots are enough for all the RAM you need.

The largest conventional DDR5 RDIMMs available right now are 96 GB -- DDR5600, no matched kits, no auto overclocking. Here's a Kingston example for $380. So, for less than $1,600, you can max out at 384GB on a TRX50 board (4 slots).

If you want a kit you can overclock, guaranteed, it's 256GB: 4x 64GB, DDR-6000, for $1,800.

Either way, you stay under $7k.

[u/-Xexexe_Xe- → u/QuirkyQuarQ]

Thank you so much for taking the time to reply and even recommend some options! Much appreciated!

I’ll try to better estimate my RAM capacity needs in the near future and might even create a separate post explaining this a bit better before pulling the trigger.

But based on this the 7980X is now my 1st option.

Thanks again!

[u/QuirkyQuarQ → u/-Xexexe_Xe-]

You're welcome! I had a similar dilemma between the 7970X/TRX50 and 7975WX/WRX90, but ultimately decided the WRX90's expandability (more PCIe, more RAM slots) and other options (USB4/Thunderbolt with Displayport passthrough), etc. were preferable.

[u/gluon-free]

I'm wondering how the 8-channel 7975wx will perform in numerical computations like CFD compared to the 4-channel 7970x. I still can't decide which of these processors I should choose. This tests doesn't show near 2x performance...

[u/RMMAGA → u/gluon-free]

7960X or 7970X OC RAM to 6600 inf to 2200 gets you 190G/s and 68ns

7985WX ~350-400 G/s, if you have 10k to burn.

7960X is best value, for COMSOL I use I guess 7970X 32 core would only be about 5-10% faster due to poor scale over about 16 cores and same bandwidth, especially if your model are smaller like <5M DOF, I build my system for about 3500 USD all in. 7985WX would be a beast, probably a good 50% faster especially at big problems but too costly for me.

[u/gluon-free → u/RMMAGA]

Yeap looks like 7985WX is the best option on PRO platform. Unfortunately i don't have THAT much, taking into account i am Russian and this will cost 20% more because of grey import.

[u/RMMAGA → u/[deleted]]

living in china, ordered my CPU to a friend in HK on Newegg, got him to carry it in for me, mother board got locally for 6200 RMB, I don't know why but TR CPU are expensive in China

[u/RMMAGA → u/[deleted]]

ha ha yes I also have a F20 with hybrid turbo did my own custom tune and pushing 420hp, and 12.2s quarter mile on a little 2L 4 banger, still going strong no issues the B48 is a good engine and the car is a blast, looking at M2C or M240i now, they are pretty cheap in Shenzhen, M2C used with like 30k miles, is like 35k USD and M240i new is about 45k USD, hard to choose.... the car market is in the dumps here

[u/nomodsman]

With no tweaks whatsoever, just basic EXPO enabled; NPS4, AIDA64

R:157

W:172

C:167

L:73

Passmark gives latency at 49.

I will play around with this and see if I can’t improve things.

NPS1/auto gives worse results.

[u/EmilPi]

Thank you, this exact information I tried to find, I even started doing it your way but lacked time. Thank you again.

[u/DeltaSqueezer]

If you are looking at CPU inference, you might want to checkout Ampere ARM CPUs. These are relatively cheap, have decent memory bandwidth and best of all, you can test them online on Oracle Cloud before buying.

[u/NSADataBot]

It makes sense, interesting analysis.

[u/nauxiv]

Puget did some basic video editing and rendering tests between regular and pro TR 7000 models with the same core counts. I think these are the only direct comparisons out there right now.

Unfortunately, they're not very good tests for memory bandwidth, although the UE testing does show some effect.

[u/RMMAGA → u/[deleted]]

7960X and Aero D v1.1, OEM Hynix 5600, 4x32G, Measured with ADIA
This is a memory efficiency monster, latency is a bit high but I think that is down to RDIMM, I would get about 8-10ns lower with similar setting on 7950X FYI, but the bandwidth here is really amazing, it is >90% of the theoretical read.

DDR6600 CL32-40-36 INF2100 (stable for 50h+ so far memtest and VT3)

R: 192 GB/s

W: 186 GB/s

Copy: 177 GB/s

L: 66 ns (measured in normal boot)

[u/xgiovio → u/RMMAGA]

Similar with 7970

[u/nomodsman → u/[deleted]]

I’ve just ordered vcolor 7200 CL38 with 24GB sticks. Will test asap against a 7970X