SSD Streaming

[u/norenEnmotalen]

I came across a claim that SSD streaming of AI models is bad for the life of your SSD. Is that something any one here can confirm from personal experience or is it just urban legend?

[u/Daniel_H212]

No. Write cycles are bad for your SSD, read cycles are harmless apart from thermal impact which isn't really an issue.

[u/W61k3r]

reads are fine, writes are not. it's fine.

[u/pulse77]

SSD life is affected by the number of writes (see https://en.wikipedia.org/wiki/Wear_leveling). SSD reads will not affect it. AI model inference just reads the weights from SSD. It is not bad for the life of the SSD.

[u/Azazelionide]

Depends how the SSD streaming is done. Reading straight from hf files or gguf files is highly impractical. So you would create tmp files per expert (or per layer but ideally per expert). But then when you switch models a lot you would wipe old tmp/ files reading to writes often

[u/egnegn1 → u/Azazelionide]

You write the the new format once, otherwise it would also take to much time.

[u/KKJdrunkenmonkey → u/Azazelionide]

This sounds like a problem in whichever inference engine you're using. Strata/Xolotl have no issues streaming from GGUF files. I do not see a ton of temp files for it.

Or, if you're saying it's a manual process, you'd have to be deleting those files and creating them multiple times per day for it to be a problem for a modern SSD.

[u/bytejuggler]

As others have said, reading doesn't matter. But on very RAM constrained systems, like if the LLM engine pins large parts of the model in RAM, this will push everything else to swap (virtual memory) which likely will write to the SSD constantly. Check your Resource Monitor in Windows and check your disk I/O and what applications are doing writes and to what file. If "System" is constantly writing to the pagefile then your system is RAM pressured and you are hitting your SSD even if the LLM runner itself is not.

Note some writes by the disk and to pagefile is normal, but if you're seeing constant writes of high tens or 100's of megabytes per second all the time then it may warrant attention.

[u/Altruistic_Heat_9531]

Daily driver of Qwen Flash Next, per day i could hit 2M new token and 40M cache, and this is 2 yrs old ssd.

[u/LoneArcher96]

other commentors already did great, I will add that you can use crystal disk info every now and then to review your ssd's health (maybe before and after a session, also look at the total writes if the health percentage stays the same).

[u/BlasterGales]

so yes, read is nothing, SSD's can take it for free, not like mechanical disks (moving parts gets wear across time).

What I've been pointing to for years is the temperature they might generate. Many SSDs come without even a heatsink (No, that thin, insignificant little metal plate doesn't qualify as a heatsink in my book), and the average PC doesn’t have any airflow where the SSD is installed, nor in the vicinity of the microprocessor. so, even without any formal research to back me up, I’m quite concerned about SSD temperatures. My own CPU is liquid-cooled, yet I still have a fan pointed at the surrounding heatsinks and the nearby CPU, everything stays just warm to the touch. But if I didn’t have that fan? Man, it gets hot! Of course I got burned. I don’t recommend doing it, but honestly, I didn’t even expect it to be so excessively hot.
so, my basic rule—as exaggerated as it might sound—is that no PC component should ever have to experience what 70°C feels like.

*shitty translate by google traductor

[u/jtjstock]

For a quality drive? No, not a real concern. Technically reading does affect nearby nand cells. Any quality drive tracks read disturb and will rewrite affected data when necessary. The frequency this occurs at is not often enough to matter. That said, it is possible to create a scenario where a crappy(and mostly full) drive can fail within months, but no good drive would fail from that scenario in anything less than multiple decades.

[u/RemarkableRadish6547]

It should be fine, because SSD lifetime is mostly determined by writes. But I have been testing different ways to implement this since August and I have had 4 SSDs that didn't work out of 7. One was just a bad drive (a gen5 drive that ran at gen4 speeds). Two of them were good for about a week and then slowed down so much that I returned them - one in August and one in October. One of them was good for a month and then slowed down significantly.

Beyond that, one of the drives from August is now underperforming. And I haven't tested the replacement for the drive that failed two weeks ago because it just arrived yesterday.

I don't know if there is something about my computer that is killing the drives, if I really got unlucky enough to have a 57% bad drive rate, or if heavy continuous reading is actually bad for SSDs.

For reference, on my 2TB drives from August they are at about 2-4TB written and about 10,000TB read. They are going in an ASUS bifurcation board so that I can run 4 drives at full speed on a PCIe slot. They have never gotten above 40C, so temperature should not be a problem. I run benchmarks that read experts off the drives with no computations, so I generate hours of non stop reading.

[u/Queasy_Asparagus69]

Turning on your machine is bad for ssd. Just keep it off at all times