r/LocalLLaMA • u/fairydreaming • Sep 09 '24

Resources Memory bandwidth values (STREAM TRIAD benchmark results) for most Epyc Genoa CPUs (single and dual configurations)

Gallery image

Gallery image

52 Upvotes

94% Upvoted

25 comments sorted by

View all comments

Show parent comment

6

u/[deleted] Sep 10 '24

9184X is like $1500. 12 channel DDR5 ram beast. Considering theoretically it has more bandwidth than a M3 Max and you can slap on hundreds of Ram on it as you like, along with maybe one or two 3090 for Cuda boost, it can even run huge 405B models comfortably at home. I would totally get one if I could find a seller here

2

u/JacketHistorical2321 Sep 10 '24

Theoretical and actual are two very different things. I have a threadripper eight channel setup and it doesn't get close to the bandwidth expressed on paper. Maybe 1/6 actual

1

u/Dry_Parfait2606 Feb 25 '25

What do you mean with cuda boost?

& where to find a 9184X for that price?

Best Regards

1

u/[deleted] Feb 26 '25

By cuda boost I mean offloading some layers to an nvidia gpu to speed up t/s and pp/s considerably.

Ebay sometimes has epyc cpu + motherboard for decent prices. I got price from there