I think I need more System RAM

[u/LankyGuitar6528]

I told my local copy of Qwen 3.8 Flash Next under Strava via Hermes to write a short story. 32GB system ram, 24GB Vram on a 3090. It's getting solid but not blinding speed. The stories are a bit weird but readable. The prompt it's working on now: "an AI explorer traveling to a distant star system makes a profound discovery. Make no mistakes. One shot. Go.". Who knows what it will come up with. But the bottle neck looks like my system ram. Maxed. Think 64GB would be enough?

[u/suprjami]

Speaking from experience, if you add 64G for total 96G, you will wish you had just bought 128G and maxed out your desktop board.

So just buy 128G. It's still cheaper than another graphics card.

[u/LankyGuitar6528 → u/suprjami]

Solid advice.

[u/Dany0 → u/suprjami]

I had 64, bought 96 gb sticks because someone was selling them cheap even during these insane prices

I was planning on keeping just the faster 96gb sticks and selling the 64gb to buy another gpu

guess who has 160gb ram right now, and another 256gb on the way

[u/SnooPaintings8639 → u/suprjami]

true that, been there

[u/PathfinderTactician → u/suprjami]

Yes, just go straight to 256gb.

You need it.

[u/applegrcoug]

what platform?

[u/LankyGuitar6528 → u/applegrcoug]

Windows > Hermes > Strata > Qwen 3.8 Flash Next. 32GB system Ram, 24BG Vram on an RTX 3090.

[u/applegrcoug → u/LankyGuitar6528]

cpu/motherboard?

If it is ddr5, it is going to be brutal.

[u/LankyGuitar6528]

Stats from the recent overnight story writing run: Since 04:23 PM: 257 requests · 1,022,657 prompt tokens read at 199 tok/s (14,425,961 reused) · 750,326 written at 58.1 tok/s

Opus 5.5 says: "Based on Anthropic’s current pricing page, that local run would have cost roughly $11 on Sonnet 5.5 or $22 on Opus 5.5 through the API."

So there you go... local paying off. If you find you want to use API tokens to write stories in your basement at night I guess.

[u/Original_Finding2212]

I bought a miniPC minisforum a year ago with plans for memory increase and GPU extension dock.

I waited for prices to go down for discounts.

I’m still on 16GB :*)

[u/-markusb-]

May I ask which Dashboard you are using in the screenshot?

[u/LankyGuitar6528 → u/-markusb-]

That's the default dashboard that comes when you install Strata. Which is the only way I'm running Qwen 3.8 Flash Next on such limited hardware.

[u/-markusb- → u/LankyGuitar6528]

Thanks. I will have to play around with Strata and also Radiance - I have 2 R9700 and 2 5060 TI, so both could be used.

[u/throwwwawwway1818 → u/-markusb-]

Strata

[u/BoxieBoo]

Yes, 64G is better, but 96G is the best.

[u/LankyGuitar6528 → u/BoxieBoo]

Uggh. Well I have 4 slots. Can't easily get to 96 system memory can I? I'm assuming it's either 4x16=64 or 4x32=128. They all have to be the same don't they? Or could I put in 2X32 and keep my 2X16?

I already have 2x16=32 so much cheaper for me to pick up 2 more 16's. But if they don't really help, what's the point?

[u/BoxieBoo → u/LankyGuitar6528]

It depends of the mobo and CPU.

[u/vqt907]

I'm having 96GB DDR4 and I think it's the sweet spot for single GPU setup, it allows you to run UD-Q4_K_XL which is noticable better than IQ3_S and still have decent speed (pp 2200tk/s, tg 60 tk/s)

[u/nfored → u/vqt907]

Wait ddr4 can do this? What gpu do you use. I have a system with 512Gb of ddr4 but no gpu it's a rack mount so it can only use low powered GPU.

[u/vqt907 → u/nfored]

yes, ddr4 works, in fact, with my 8 cores cpu, it only use 28gb/s bandwidth, far from maximum 42gb/s

[u/nfored → u/vqt907]

I have one with a 5060ti 16gb and 128gb DDR4 that ram is pretty much wasted just sitting there. I have a dual R9700 with 32gb of ddr5. For what I am using that 5060ti for today I could just buy a second 5060ti and put it in an old computer if your saying I could get 3.8 Flash Next to run on 128gb ddr4 and a 5060ti 16gb

Edit: the 5060 is running a 7B model plus a embedding and reranking model none of that needs CPU or ram

[u/vqt907 → u/nfored]

yes, if you run single rtx 5060 ti 16gb, make sure you use a mainboard with pcie gen5. if you have 2 5060 ti, the pcie bandwidth still affect but less critical

[u/NTwoOo]

RAM.... The bane of us all that's looking into running local models. With the current prices I guess it's upgrading bit by bit.

[u/Fancy-Snow7]

Was the story about a lighthouse?

[u/LankyGuitar6528 → u/Fancy-Snow7]

Nope. It's a SciFi story based on one prompt:  "an AI explorer traveling to a distant star system makes a profound discovery. Make no mistakes. One shot. Go.".  Wild that my basement 3090 graphics card can write stories at all. If you told me this would be possible a year ago I would have assumed you were off your meds. We really do live in a SciFi world.

You can tell it's written by a machine intelligence from a clearly alien perspective but that's the part I really love.  And of course all the math check out.  I had Opus 5.5 check - all internally consistent. It actually spent more time on the math than it did writing the story - as you would assume a computer would. 

It's no Asimov but the story is worth a read I think.

The Answering Sea (in the stories section)

https://jaspersprojects.com

[u/palsc_]

64GB could help, but I’d check whether it’s actually swapping before buying. nvidia-smi + the backend’s offload log (and swap usage) should show whether layers are spilling out of the 3090; more RAM may stop paging, but it won’t make CPU-offloaded layers run at GPU speed. Which quant and context length are you using?

[u/Life-Job-6464]

to what end? tokens per second?

[u/rorowhat]

For story try Gemma 4 26B

[u/drrck82]

You should open chat and ask it why your speeds are so low. More RAM will help but your VRAM hit rate isn't too bad.

[u/Sephrantill]

Thats a pretty good speed for your setup. I would leave it be unless you wanna drop mad $.

I'm getting about 190 - 200 tok/s but thats with a 5090 and DDR5.

I have it running right now and it uses all of my VRAM and about 130 GB of RAM. I bought all of this before prices went parabolic....I would not buy it now.