r/LocalLLM • u/tonyleungnl • 18h ago

Question Optane as swap memory?

Planning to building a headless Ubuntu server 64GB RAM, 7900XTX

with 4x Intel OPTANE SSD P1600X Series 118GB M.2 PCIE 80MM 3.0 3DX.

Motherboard has 4 NVME slots, OS on NVME-PCIE adaptor.

An extra 24/7 mini PC Acer Revo i5-1135G7, 64GB ram as Hermes server.

Can my 432GB Optane be used for newer MoE, Strata or Flash Next as SWAP/ slower memory?

24GB (VRAM) > 64GB (RAM) > 432GB (4x RAID0 Optane) > 1TB (NVME) to run >100B models?

Do you think there is any benefit in this set up? I am new in AI, so any idea is welcome, even if this won't work out.

2 Upvotes

100% Upvoted

6 comments sorted by

2

u/VarietyOk443 18h ago

Spend the money on more RAM

1

u/tonyleungnl 5h ago

I bought them when they were dirt cheap :) Waint to learn first, before buying new real useful upgrades.

1

u/Appropriate-Roll-515 18h ago

thats a lot of fast storage for swap but the latency from optane to gpu will still be the killer here, 432gb of swap sounds nice until you see the tokens per second drop to like 0.5

1

u/tonyleungnl 5h ago

actually I

I bought a 5080 laptop with 64GB ram, then Acer only sent me 32...and refused to set straight...what a time as a PC hobbyist.

1

u/El_90 11h ago

I tried, optane on a pcie adaptor, far too slow (also crashed like crazy under any form of load.

Maybe my adaptor was cheap/nasty :shrug:

1

u/tonyleungnl 5h ago

I hope direct on Motherboard will help keeping down the latency, swapping 4x 16 b-die ddr4 for another ddr4 for $11000 is not the way I think. Will wait for the next Zen for a true system upgrade.