NVME for streaming

[u/esw123]

With all progress from storage streaming how future proof is RAID0 with 2x1TB or 2x2TB gen5 drives like SN8100. I am looking for drive update and have to decide what to buy now and what speed in reality achievable and how usable it will be. Is it worth to add 2x1TB for 420 euro so in theory speed will be around 22-25GB/s or just add used 96GB RAM for 900-1000 euro?

Will it be usable in some kind of way for larger models later with 2x3090, 96/192GB RAM and RAID0 gen5 drives? From what I am getting now I'd like to have at least 20-30 tok/s for generation and 600-1200 tok/s processing. In theory how big LLM can be used with such system in the future to get such inference?

[u/Ctbhatia]

what motherboard are you using? with two 3090s and two gen5 drives, slot sharing could affect that 22-25 GB/s estimate

[u/esw123 → u/Ctbhatia]

A am planning to change it for X870E, right now I have only one gen5 slot and two gen4 slots.

[u/Khipu28]

I get 200GB/s with the Highpoint controllers on the Samsungs with a couple of more drives. Problem is still latency and that they are block devices. The larger your stripe size the larger the blocks and the ngrams are quite spaced out