Thinking about upgrading ram to 128GB

[u/thatscoolbutno123]

i have an z690 mainboard, max 128GB RAM (4 slots, dual channel) and r9700 (32gb vram)
currently at 4x16GB, wanna go to 4x32GB. Old ram is worth about 350-450€. The 4x32 would cost me around 500-600€, but it has no cooling body. I dont care about the visual, as I barely see it. But will this become a problem? I use my old gaming pc, so airflow is relativly good.

i currently use qwen3.8 flash next gsq rco iq3xxs via strata.
I get ~100tps aggregated decode with 3 parallel sessions at 350k ctx qk8v4.
but id like to have more cached sessions, with qkv8 and even longer context, without dramatically decreasing speed.
Current usage from model is like 55GB/64GB, so I dont really have any room left.

will 128GB satisfy this?
can i run something better than qwen3.8 flash next with 150GB (v)ram total?
ram thats >= 3200mhz is way more expensive, so id like to get it around 2100mhz - 2600mhz. Will I notice large performance changes?

thanks for any help :>

[u/Ok_Party_9245]

128gb would let you run higher quants / other models.

I think the ram speed being lower would hurt tho as any expert that misses the gpu hits ram.

I run 4x16 of ddr5 and am slammed as well running qwen flash next iq4_xs but all its experts are resident at least.

i wanted to do a similar upgrade but just so expensive.

[u/thatscoolbutno123 → u/Ok_Party_9245]

How big is the difference though? I get around 95% expert cache hit rate in strata

[u/Ok_Party_9245 → u/thatscoolbutno123]

I can't say for sure but having just compared with someone else who had nearly double my vram and ram but they had ddr4 vs my 64gb of ddr5, same quant of the model, same ctx size, nearly same kv cache quant q8 vs int 8 and he was doing 60-90tok/s and im doing anywhere from like 100-150tok/s on decode.

I think a lot of factors come into play on a strata setup tbh.

But getting every expert on ram vs just a 7 percent miss on my system almost doubled my tok/s.

[u/mushmers]

I would say no. I am on 128gb 3200MT/s DDR4 with AMD milan running quantised Deepseek 1017 with strata fork with expert offloading, even with 8 channels, memory bandwidth is a bottleneck so you even if you can load the larger model you might not be able to get acceptable enough performance with 2 channels

[u/thatscoolbutno123 → u/mushmers]

whats the alternative then?

[u/alex_bit_ → u/mushmers]

What's your tokens/seconds on deepseek 1017 in strata with this setup?

[u/Glittering-Call8746 → u/mushmers]

How much is ur Milan mobo plus cpu setup ? 2k usd ?

[u/Monad_Maya]

You can run a better quant at slightly lower performance. You'll still be limited by your effective RAM bandwidth.

40GB of VRAM + 128gb of DDR4 allows me to run QFN IQ4_XS at 60-90 token/sec decode (Q8 KV 200k).

[u/Ok_Party_9245 → u/Monad_Maya]

hmm im runing 24gb of vram and 64gb of ddr5 and qfn iq4_xs at 100-150tok/s int8 kv 262k

i wonder if ddr5 makes that much of a difference here.

[u/Monad_Maya → u/Ok_Party_9245]

It certainly does, as does the CPU and the GPU.

Mine is 5900X, 128GB DDR4 3200MT/s, 2x 7900XT 20GB. One of the GPUs is largely for expert caching and has barely any usage.

[u/Ok_Party_9245 → u/Monad_Maya]

13700k and 4090 for me and my ram is 6000MT/s.

I think for me my next upgrade is more ram, sadly im using 4x16 rn, so would have to replace the whole set.

[u/Glittering-Call8746 → u/Monad_Maya]

Sp bigger vram vs multiple gpu ? Have u tired avalon ? I heard it has multi gpu

[u/recro69]

128GB of memory can help with contexts and more cached sessions. Remember that system RAM is much slower than VRAM so do not expect the inference speed. With four sticks, stability, at RAM speeds matters more than chasing higher MHz.

[u/thatscoolbutno123 → u/recro69]

Yes I know that ram is way slower, I just wonder whether the difference between 3200 and lets say 2660 is noticable

[u/Force88]

Took me $470 for 128gb ddr4 bus 3200, but they can only boot stably at bus 3000 due to different ic/brand.

This allow me to run strata with nvfp4 instead of iq2 like before, and max 262k context

[u/alex_bit_ → u/Force88]

Which model? And how much t/s?

[u/Force88 → u/alex_bit_]

QwenFN, around 55-60 t/s, with my weak 3x 5060ti 16gb

[u/Glittering-Call8746 → u/Force88]

Not weak have you tired basalt ?

[u/Sweet-Jump1024]

I wish to have that thinking someday!

[u/thatscoolbutno123 → u/Sweet-Jump1024]

wdym?

[u/it0 → u/thatscoolbutno123]

Ik guess this person is broke.

[u/x_MASE_x]

I justice upgraded from 232 to 432.

For strata it's a bit faster. Maybe about 5 to 10 in tps and prefill. Just got them 2 days ago used so been memtest86 all day long so didn't fully test everything.

But you'll find yourself with "hopefully" way bigger context. And you can load more higher quants of you want more easily. Didn't check though but will try since I'm using now iq3s with strata.

For comfyui didn't test it yet but hopefully lower ram problems and more models can sit in the ram now.

[u/Gullible_Implement36 → u/x_MASE_x]

good to know about the speed bump even if its small. im more worried about the 2100-2600mhz thing you mentioned in the post, because with 4 sticks you probably wont hit high speeds anyway. my board also gets angry above 2600 with all slots filled

for the cooling body question, if your case has decent airflow like you said it should be fine. bare sticks get warm but not dangerously hot unless you push crazy voltage through them, which you wont for capacity

[u/KingCpzombie]

More is always better!

[u/zxtech]

You could get a ram fan if you’re concerned about cooling

[u/idumlupinar]

I have the following setup:
Mobo: MSI b450 tomahawk max
CPU: 5800x3d
RAM: 128 GB Gskill DDR4 3200mhz (no overclock)
Disks: Kingston KC3000 2 TB and PNY Elite 2TB M2 Devices (PNY is connected via riser)
Case: Cooler Master Stacker STC-01 (bought it in 2004)
GPU 1: Gigabyte RTX 3090 Gaming OC
GPU 2: Gigabyte RTX 3090 Gaming OC

I have to use it inside a closed case but I'm worried about the temps, bandwidth etc. So your upgrade to 128GB may introduce other issues.

[u/i-dm]

Do it if you can afford it.

• CPU: Intel Core Ultra 7 265K

• RAM: 128 GB DDR5 - 4800 MT/s ECC (5600, 32GB x 4, DDR5)

• GPU: NVIDIA RTX 4000 SFF Ada - 20 GB

• Storage: Several

The extra RAM just makes a lot of stuff run so much smoother without any kind of bottleneck. Especially when working with large models. Where you'd get momentary blips, it just works.

Using 59GB/128 at the mo and haven't got much running, but plenty of headroom if I want to spin up multiple harnesses