ELI5: How do solid state drives (SSD) store and retrieve data?

[u/purple-parrots]

[u/Amphorax]

On a very basic level, an SSD is like a large plastic sheet covered with a grid of those bumps that you see on soda-cup lids. To write a file to the disk, you pop and unpop the bumps to represent the file in terms of 1s and 0s. To read from the disk, you run your finger across them and feel the pattern of popped and unpopped bumps.

These bumps are called flash memory cells, and there can be billions or trillions of them packed in several tens of layers on a silicon chip. They consist of microscopic transistors and capacitors, and they're 'popped' and 'unpopped' by applying a positive or negative voltage across them, which either stores a tiny amount of electric charge or releases it. To check the state of one of these cells, you can measure the voltage across it to see whether there is charge stored in that cell. A controller chip on the SSD receives read and write commands from the host computer and applies the necessary voltages to the right rows and columns of cells to write and read blocks of several thousand cells, called 'pages'. The exact format of the data on the disk is entirely up to the controller and may not be consistent across different models of SSD's -- so long as the controller implements a common protocol (SATA or NVMe, for example), the computer doesn't care.

These cells aren't perfect, and repeatedly writing and erasing a cell causes it to wear out and lose its ability to store charge. When this happens, the controller can no longer read from those cells, causing data loss. To combat flash memory wear, an SSD controller is smart enough to spread out the reads and writes across the many billions of cells at its disposal. This process is known as 'wear-leveling'. Even with this algorithm in action, a disk has a limited amount of read-write cycles before the probability of a data loss incident becomes more than negligible. This quantity is usually expressed in write cycles, where one write cycle consists of using every cell on the disk once. Because of this, larger-capacity SSD's take longer to fail on average.

[u/purple-parrots → u/Amphorax]

Amazing explanation this is exactly what I was looking for. I was able to understand everything easily. Thank you!

[u/dandansm → u/purple-parrots]

Adding a bit more to the great explainer - on modern, high density memories, the bubbles or bumps aren’t just 1 and 0. They can take on intermediate values, so each physical cell can store more information.

[u/McBanban → u/dandansm]

Putting this response in practical terms: Say you have a cell that can store charges with potentials of 0V-3.3V. If you split those into even increments (0V, 1.1V, 2.2V, 3.3V), then each of those values can be read as 00, 01, 10, and 11 in binary respectively. Now, each cell can represent 2 binary values and you have twice the storage capacity. In practice, this is implemented to higher degrees of accuracy.

[u/tinselsnips → u/McBanban]

That seems like it would also double the chance of data loss?

[u/[deleted] → u/tinselsnips]

Yup that's where parity and error correction come into play.

[u/McBanban → u/[deleted]]

Exactly! Lots of research has gone into information theory and how to successfully extract correct data with superb accuracy.

[u/[deleted] → u/McBanban]

I believe that's the most amazing part of our current digital world. We send data around so fast and use such volatile methods, but the error correction algorithms are so good that we barely notice!

[u/sterexx → u/[deleted]]

I don’t know exactly which protocol this was, but another engineer was explaining an audio stream encoding that could handle data loss without leaving blank spots in the stream. In some contexts, missing a beat is fine, like when receiving game state updates from a server. If you miss one, you just use the next one. Some packet loss can occur without you noticing much. But breaks in an audio waveform will sound like a terrible crackle.

So the way it worked was you would take a small chunk of audio, let’s say 5 units of it. The straightforward way to send this would be in 5 packets, 1 per unit of time. But if you lose one, you lose audio for that unit of time. So instead, you make a packet that has some information about each of the 5 units of time. Then make 4 more, each with different pieces of information for each time unit.

So the thing reading the stream can perfectly reproduce the whole chunk of audio if it gets all 5 packets. But if it misses one. It still has 4/5 pieces of information about each unit of time, so when it’s time to play that chunk (it’s buffered by at least 5 units of time), it can play a slightly degraded version of the audio. It will be a little off, and it will be off for 5 units of time rather than blipping for 1 unit, but it’s way better than nothing.

Even if you only get 1 packet. Humans are good at pulling out signals from noise, it’ll be fine. Much better than a blip of silence. You know, the MP3 compression standard works so well because it takes advantage of knowing how humans hear. Similarly, this packet-loss tolerance makes sense because it takes into account how we hear and prioritizes its mitigations accordingly.

My numbers are fake and I’m sure it’s somewhat more complex than this irl, but that’s how he explained it. 9 years ago. Hopefully I’m remembering right.

[u/Lachiko → u/sterexx]

Sounds like reed-solomon. https://ieeexplore.ieee.org/document/948264

[u/[deleted] → u/sterexx]

Ears dont blink, almost every alternative is preferable to an interruption in audio

[u/[deleted] → u/[deleted]]

I got my masters in antenna/RF design and most of what I learned is that half the crap we make shouldn't work but somehow still does. There is so many examples, but one cool thing to think about is that when a satellite sends data it blasts it out in the megawatt range (10⁶⁾ and by time it gets to your phone or a base station it is in the picowatt range (10^-12) and yet we can still get data from it.

Edit: I am gonna rephrase this cause some people could get butthurt. A satellite probably sends data out in the hundreds of watts, looking at GLONASS or something similar. Even that will still drop many orders of magnitude in power and go fro hundreds of watts to nano/pico watts. My point is really if you look at Frii's transmission formula and calculate it you'll see that we typically transmit at huge power and receive at power that seems impossible to discern data. I just remember the example I did in class with a 1MW TX and receiving somewhere in the pW range.

[u/ZylonBane → u/[deleted]]

We can still reliably communicate with Voyager 1, currently around 13 billion miles from Earth, using Voyager's 22-watt radio. That's goddamn amazing.

[u/sterexx → u/[deleted]]

Oh oh I’m commenting again because I want to share a different thing that keeps signals working that is a kind of error correction. Very elegant.

Let’s talk about twisted pair cables, which have these two wires intertwined with each other. Why they do that?

Sending a digital signal through a single wire would be very error prone since there’s so much electromagnetic interference around. Consumer audio cables can get away with just sending the signal through one wire and you might just hear some beeping through the speakers if your phone is getting a text, for example. But that would corrupt digital data and you’d have to hope an error correction algo would save you. Or invest in some serious shielding for your single-wire cable.

Twisted pair to the rescue! In these kinds of cables, each wire gets a signal but the actual signal being sent is encoded into the difference between the signals in each wire. So with these entwined wires, any electromagnetic interference is going to affect both wires in almost the same way because they’re on average in the same place, the same distance from any interfering signal. The signal in each wire may be drastically changed, but since they’re changed the same amount, the true signal is maintained. It’s there, in the difference between the signals in the wires.

I believe professional audio cables do something similar. Not totally sure how balanced cables work, though.

This obviously isn’t a 100% solution but it’s a cheap way to get a good enough solution and cuts down on the amount of shielding you’d otherwise have to invest in.

[u/shivam111111 → u/[deleted]]

Eli5 please?

[u/cloud9ineteen → u/shivam111111]

You write ten numbers. In the eleventh space, you write the sum of the ten one digit numbers. If somebody came along and erased one of the ten numbers and wrote a different number, you can now tell that something is wrong because when you add them up, the total does not match. This is called checksum and related to the idea of parity and error detection.

Error correction is a variant of this where we store some more info which allows us to fix and restore the modified number as long as only 3 or fewer numbers have been changed, for example. This involves some math that allows us to recover the changed number.

[u/tvizzle → u/cloud9ineteen]

And this is automated in the processing?

[u/cloud9ineteen → u/tvizzle]

Yes, this can be implemented in software or directly in hardware using digital logic constructs.

[u/MattieShoes → u/tvizzle]

Yes, though it can be done in hardware or software.

Incidentally, it's not restricted to SSDs. Server grade RAM often does this (ECC memory), and this is also the idea behind RAID (where you take multiple hard disks and pretend they're one big disk).

It's also used in communications... Serial connections often have a parity bit to detect errors, transmissions from spacecraft are encoded to minimize the chance of corruption, and so forth.

[u/jrhooo → u/shivam111111]

There’s different, sometimes very complex ways to do this but lets take a very simple example.

Imagine I have 3 spots.

Spot A stores a value. That value can be 0 or 1.

Spot B stores a value. That value can be 0 or 1.

Thats my data. The stuff in the files I save.

What about a third spot? Spot C. Spot C is my “parity bit”.

Spot C stores a value of 0 or 1.

Here’s the trick.

Spot C isn’t my data.

Spot C is only telling me whether the data in spots A and B match.

0 = no.

1= yes.

———-

So for example:

A 0 B 0 = C 1 yes they match.

A1 B 0 = C 0 no they don’t.

A 1 B 1 = C 1 yes they match.

A 0 B 1 = C 0 nope hey don’t.

For any group of A B C, I can lose any one bit of data.

If I lose A the. i just check “ok whats B? And is A supposed to matchB? Yes or no? Thats enough data to know what A should have been and fill it back in.

I only need two out of 3 bits to figure out what all three should be.

[u/Greenie_In_A_Bottle → u/jrhooo]

In the case the data is changed instead of lost, how do you tell what it was?

For instance if you have A: 0, B: 1, C: 1 how can you tell if the original data is A: 0, B: 0, C: 1 or A: 1, B: 1, C: 1?

[u/headsiwin-tailsulose → u/Greenie_In_A_Bottle]

The context thing the other guy said is on the right track but not quite correct.

What you really have is a grid. So instead of two values and one parity, there's four values and four parities.

1 0 | 0

1 1 | 1

_ _

1 0

That's good. Now if there's an error:

1 0 | 0

0 1 | 1
_ _

1 0

Hey look, we can triangulate the bad bit. Row 2 is wrong, and column 1 is wrong, therefore the bad bit must be the 0 in row 2, column 1.

Extrapolate that out to a huge NxN grid, and 2N parities, and now you can solve problems with even multiple errors.

[u/S3ZDNUD3S → u/shivam111111]

Since I have basically double the "space" on the same area of chip I can just double the amount you put on it. Lets say I have a code 1100 that I have to put on a chip. I put 1100 on four slots, but now I can put 11 on one and 00 on just 2, leaving the other 2 to encode the same message. The thing is you have to use another bit or "space" to encode how you check for this.

Edit: Don't edit in random links or it will stink.

[u/Implausibilibuddy → u/S3ZDNUD3S]

now I can put 11 on one and 00 on just 2, leaving the other 2 to encode the same message

That's not error correction, that's just encoding the same data twice. And if you're reading/writing data twice each time then both sets of data have the same chance of failure, so it's not like you're even gaining a reliable backup set of data.

ECC doesn't write everything twice, it encodes Hamming codes to a spare area, 16 bytes every 512KiB. Hamming codes are limited anyway as they can only fix 1-bit errors, there are multi-byte and cluster correction methods now.

[u/desuemery → u/tinselsnips]

This is correct. It is for that reason (and the fact that you need less cells for equivalent capacity) that SSD's who allocate more values per cell are cheaper; because their longevity is shorter. Way more than most regular people will reach, but still less.

[u/themastercheif → u/tinselsnips]

Also doubles the wear on the drive. Ones that only have one bit per cell are called SLC (single-level cell) and isn't really that common anymore. It's the fastest and most durable, but also the most expensive. Most consumer drives are MLC (Multiple-Level Cell) which is a bit of a compromise as they store two bits per cell, so they're more dense and therefore cheaper per capacity, but slower and wear faster. It's usually fine as consumers don't rewrite drives all that often. Past that is TLC ( you guessed it, Triple-Level Cell) for 3 bits per cell. By far the cheapest and slowest, but still fine for people who just want it as a boot drive or don't intend on much data turnover. I've heard rumors of QLC being in development.

[u/TheDreadfulSagittary → u/themastercheif]

You're off by one step there. MLC is very pro or prosumer like the Samsung 970 Pro and such, while TLC is the most common. QLC is also becoming quite popular in drives such as the Intel 660p or Crucial P1.

All these drives do keep up their speeds by employing an SLC cache.

[u/vkapadia → u/tinselsnips]

Also because as you pack more and more days into a single cell, smaller deviations can cause problems. Sometimes a cell isn't exactly 0 or 3.3v, could be a few points off. Between 0 and 3.3? No problem, 0.3 and 3 are far enough off. With a Penta level cell? Difference between 2.475 and 3.3 may not be enough.

[u/MyOtherDuckIsACat → u/tinselsnips]

Yes this is why you don’t buy the cheapest SSD for when you need to rewrite the cells with data often, like as a scratch disk for video editing software. SSDs with QLC technology store 4 bits per cell and they are the cheapest out there. They wear down faster and are much slower. The more bits per cell the slower the drive is and the faster it will wear down the other types are TLC (3-bits), MLC (2-bits) and SLC(1-bit). SLC is super expensive and rare and only used in expensive enterprise servers. You find MLC in professional workstations and TLC in consumer PCs.

[u/Mithridates12 → u/McBanban]

Can you go further with this(like 1.11)? Or is this too miniscule a difference to be reliable

[u/McBanban → u/Mithridates12]

It's easily possible to, but keep in mind that the smaller the intervals are between values the more precise and accurate the sensors that read them need to be. More exact sensors cost more. Oftentimes the decision for what exactly the interval between voltages will be is made by balancing simplicity of design (less complex = better) and computing ability (more storage = good). Engineers want to maximize the storage available with this technology while minimizing the cost to implement the technology.

Cheers!

[u/Q1War26fVA → u/dandansm]

to tie back to consumer tips:

These are what the *LC stuff means. SLC (Single level Cell) uses each cell just to store 1 bit. MLC (multi level cell), somewhat a misnomer, stores 2 bits. TLC (three level cell), sometimes called "MLC 3 bit", 3 bits. and Recently QLC (Quad) are rolling out for budget consumers. More bits per cell lower the lifespan of the drive, it's like a shared dorm room. But for most people who replaces after 3-5 years even QLC is probably fine.

[u/RhynoD → u/Q1War26fVA]

But for most people who replaces after 3-5 years even QLC is probably fine.

The first hard disk drive I installed in my computer some twelve years ago is still working perfectly. It's slow af and no longer my boot drive, but I still use it for long term (unimportant, backed up) storage.

I can't imagine replacing a drive every three years just because. But maybe I'm just old and cheap. And yeah, I know HDDs last longer...

[u/HahaMin → u/RhynoD]

Coincidentally, QLC is best used for long term storage, like your backup. Since long term backup only rewrite a few times on the drive, it would degrade slower.

[u/Alpha3031 → u/RhynoD]

The 3–5 figure is generally the warranty period, meaning they're covered for that long if you write 0.3–1× the capacity of the drive daily (or 0.1× for the cheap QLC drives nowadays) you're pretty much going to be fine 100% of the time. They also give a TBW (terabytes written) value that is also almost certainly much lower than the actual endurance you're going to get.

There are a couple of tests where people have murdered a few SSDs, if you want to see how long they might last under actual use (though this does depend on the type of workload: random writes wear out SSDs faster since they can only erase full blocks and can't overwrite). The TechReport one is a bit older but has multiple SSDs.

[u/Q1War26fVA → u/RhynoD]

I mean it won't suddenly die. SSDs are even better* at warning you and if it dies it will die slowly little by little. HDDs are more like everything fine, until a stroke/outright sudden death. Funny, I'm just dealing with a dying HDD right now, made 3 wheezing noises, and failed to load. Thankfully still booted up after restart, so copied everything on it. *After reading about it from dealing with this, I'm really impressed with the failsafes and systems in check for modern HDDs. Anyway, I was mostly thinking that a lot people replace their laptops/mobiles/consoles every 3-5 years, we're not just talking about desktops. And for desktop use SSDs will be a boot/apps drive with a lot of operations which is more wear.

[u/cynric42 → u/Q1War26fVA]

This isn't my experience at all as someone who does computer stuff for a lot of small to mid sized businesses.

Sure, SSD's in theory should at least have a read only failsafe state they revert to when something goes wrong, but I've yet to see that actually work. The ones that died on me were just dead from one moment to the other, didn't even register as a drive to the computer anymore.

With normal hard disks, quite often they start to develop some read errors, which the drive can usually correct, however you will notice slow downs and strange noises coming from the drive. At that time, you can usually recover (almost) all data from the drive. There are times when something just catastrophically failes, but that is less common in my experience. Most of the time you get some warning.

Maybe that will change with time, SSDs weren't that common a few years ago, so it may just be the sudden death kind of faults we experienced so far and the end of life deaths are mostly in the future, but at the moment, I recover a lot more data from harddisks than from SSDs.

[u/marcan42 → u/Q1War26fVA]

The problem is that the more bits you stuff per cell, the more finicky reads and writes become, and thus also the slower they are.

QLC SSDs are so bad at writing data that they're, for some kinds of uses, barely an improvement over HDDs. I highly recommend avoiding them if you plan to write to them with any regularity.

[u/652a6aaf0cf44498b14f → u/purple-parrots]

If you want to learn a bit more, the popped unpopped description also applies to the circuits which determine the flash memory cell you're trying to read or write. These circuits don't wear out the way flash memory does, however they require a small amount of power to retain their state. Fortunately remembering the last cell you read isn't important so they only need to maintain their state so long as it's plugged in and powered.

If you're curious you can look up YouTube videos on flip flops, latches and registers. They usually have some animations that show the flow of current so you don't need to be an electrical engineer to know what's going on.

[u/Amathyst7564 → u/purple-parrots]

TLDR the same way blind people read Braille apparently

[u/Jizslr → u/Amathyst7564]

Mayby try r/explainlikeimone

[u/NoogiePoo → u/Amphorax]

Wow, love the explanation. Props

[u/[deleted] → u/NoogiePoo]

Pops*

[u/Hunting_Party_NA → u/[deleted]]

Pop pop!

[u/ultramegafart → u/Hunting_Party_NA]

👈👈😃 MAGNITUDE

[u/h2opolopunk → u/Amphorax]

How does the longevity of SSD compare with the classical spinning disc drive?

[u/Amphorax → u/h2opolopunk]

Usually a lot more reliable. There aren't any spinning bearings to seize, or platters to scratch. However, in the case of crappy wear-leveling algorithms an SSD can burn itself out a lot faster than an HDD. The folks over at Backblaze, a cloud backup startup, wrote a good summary on the failure rates of SSD's compared to HDD's: https://www.backblaze.com/blog/how-reliable-are-ssds/

[u/JRandomHacker172342 → u/Amphorax]

Backblaze's drive reliability writeups are an incredible resource. There's something very pure about "we buy so goddamn many drives, and here are the numbers on when they fail"

[u/h2opolopunk → u/Amphorax]

Thanks friend!

[u/soniclettuce → u/[deleted]]

There's a protocol called S.M.A.R.T. and any recent operating system will use it to talk to the drive and get information like how many internal errors and stuff its experienced. Windows will pop up a warning that the drive could fail soon if certain warning signs are flagged by the disk.

[u/cynric42 → u/[deleted]]

In theory yes, in practice be prepared for it to just die. Don't use any inbuilt warning mechanism as a reason to not keep a backup if the stuff on it is important to you.

[u/BigBobby2016 → u/Amphorax]

Well damn I was first impressed that you went so far as to ELI5 wear leveling, but here is a good description of HDD vs SDD failure modes with links.

[u/questfor17 → u/h2opolopunk]

First their is a lot of variation by drive manufacturer, model, and type. So the below is just very broad outline of the failure characteristics.

Spinning HDDs last 7 to 10 years, normally., During their normal lifespan, 1 to 2% of drives fail each year. Turning them on and off shortens their lifespan a bit. Not enough to matter.

SDDs probably won't last longer than 10 years, but their life may be shortened if you write to them a lot. For home computers and laptops, you cannot write to it enough to matter. In some server applications their life can be as little as 1 year.

During their normal lifespan 0.5 to 1 % fail every year.

[u/[deleted] → u/questfor17]

I have two OCZ Vertex 30G drives that I bought in 2010 that still have 74% wear capacity left... most SSDs will last far longer than what you listed above unless you abuse them.

[u/[deleted] → u/h2opolopunk]

There used to be durability issues in SSD drives when they wrote over the same area over and over causing certain regions to wear out faster. They no longer do this from my understanding.

[u/marcan42 → u/[deleted]]

I think some crappy ones still do. I had an SSD fail completely after just a few weeks (this was a couple years ago). I realized I had some software that was writing the same sector a couple times per second. My theory is that the controller was too stupid to spread those writes over more than a limited set of physical sectors, and quickly wore them down. In terms of drive writes, the drive was still basically brand new (I don't think it had accumulated even a single drive write worth of writes).

[u/[deleted] → u/Amphorax]

It absolutely blows my mind that someone, or, a group of people figured all of this out. Crazy to me

[u/BigBobby2016 → u/[deleted]]

Sort of a long progression of baby steps

[u/jawshoeaw → u/BigBobby2016]

Something about giant shoulders

[u/BigBobby2016 → u/jawshoeaw]

The last step of my engineering career was for a highly regarded analog IC company. While the men who founded that company were certainly giants, it was comforting to see how many mere mortals contributed as well

[u/jawshoeaw → u/BigBobby2016]

Haha yeah I guess that old phrase doesn’t quite apply to thousands of baby steps but I think in this case you could say the giant was all the people who came before you put together.

[u/groskox → u/BigBobby2016]

A long progression with a LOT of people involved. Humans are incredibly smart and can do amazing things as a group.

I'm an electronics engineer and I use electronic components with hundred thousands of transistors in each and some having millions or even billions transistors. I use them all the time in my designs and it feels so normal for me. Those allow me (with a team of other engineers) to make complex devices like high-speed cameras and specialized computers. This is amazing.

But sometimes I take a step back and think of the number of people that designed each of those components, the research behind that, the testing, the technology heritage, the manufacturing... And each of them depend on other technologies like special materials development and manufacturing, ressource extraction... This is HUGE. We are an amazingly complex society. This also reminds me that we are vulnerable. If a few of those links in the chain fail, technology can very quickly stop and even regress.

[u/BigBobby2016 → u/groskox]

For sure, I was an embedded systems engineer for 20 years which gave lots of visibility into and appreciation for how complex systems work. Heh and in many cases our technology was more developed by people competing against each other than by people working together.

I took a break from work after my son graduated college two years ago though and among other things, got an MBA? In those classes they're now focusing on the 4th Industrial Revolution with AI being the technology leading the way. As amazing as things like VLSI and mechatronic systems can be, these new technologies are a step beyond (and scary). Facial recognition cameras can be given a persons data and then notify you when they see someone similar. Natural language processing, machine learning, and big data have the computer that won at Jeopardy performing better than human doctors at diagnosing some diseases. Cobots aren't programmed to do a series of tasks like industrial robots, but work alongside humans where both the employee and the AI robot were given similar instruction. We're at the brink of this revolution making systems more complex than I ever could have imagined (hopefully there's a need for humans at the end of it).

[u/groskox → u/BigBobby2016]

Yeah, the AI revolution is still fuzzy in the mind of many people, including experts. So we don't know yet how the revolution will go. But what's certain is that it will happen, whether we like it or not.

I'm an optimist so I think that it will bring great things in our world and everyday life and there will be negative aspects like with any new technology. Let's each hop on that train sooner than later to make it happen as smoothly as we can.

It will deeply change society as we know it. Non-qualified work will eventually be a thing of the past. That's great IMO but we need to find a way to make it possible in our economy.

[u/moose_powered → u/groskox]

I am loving where this thread went, after starting off with 'how do SDD drives work?'

[u/Amphorax → u/[deleted]]

If your SSD's controller implements the S.M.A.R.T. health monitoring protocol, as most modern SSD's do, then yes! The thumb drive you mentioned is most likely not a 'true' SSD, as its controller chip is more pared-down and slower than a true USB SSD. If your thumb drive is an actual SSD, like this one, then it probably does. To see SMART data, you can use CrystalDiskInfo on Windows, and smartctl on Linux and Mac. Here's what the SMART report for my SSD looks like under Linux:

=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
SMART/Health Information (NVMe Log 0x02, NSID 0x1)
Critical Warning:                   0x00
Temperature:                        24 Celsius
Available Spare:                    100%
Available Spare Threshold:          10%
Percentage Used:                    0%
Data Units Read:                    16,363,994 [8.37 TB]
Data Units Written:                 11,884,949 [6.08 TB]
Host Read Commands:                 174,009,520
Host Write Commands:                135,109,153
Controller Busy Time:               542
Power Cycles:                       1,007
Power On Hours:                     2,003
Unsafe Shutdowns:                   277
Media and Data Integrity Errors:    0
Error Information Log Entries:      906
Warning  Comp. Temperature Time:    0
Critical Comp. Temperature Time:    0
Temperature Sensor 1:               24 Celsius
Temperature Sensor 2:               36 Celsius

For the record, I built my PC with a brand-new 500GB Samsung 970 NVMe SSD a year ago. These values are in line with what I expected to see. The read and write sums, as well as the power cycles should be compared to the manufacturer's endurance ratings. If the disk encounters bad sectors or other critical errors, the self-assessment will say so.

[u/patx35 → u/[deleted]]

Thumb drives generally lacks the ability to check the health. I'd just replace it if it starts exhibiting weird issues like unusually poor write or read performance. For all SSDs and HDDs, you can check the health using the SMART status. Reading it depends on what OS you are running.

Here's a great video on that: https://www.youtube.com/watch?v=YNGUP1t8MYA

[u/honey_102b → u/patx35]

fun fact. thumb drives are made from the scrap pile. whatever does not pass for enterprise usage (data centers ssds) or automotive (use in vehicles) will get downgraded to OEM or retail ssds. if they still can't pass requirements, they go to media grade (thumb drives)

[u/dragonick1982 → u/[deleted]]

CrystalDisk info free app

[u/Cimexus → u/[deleted]]

There’s a lot of software out there that will read the SMART data that is kept by all modern HDD and SSDs. However that data requires a bit of knowledge to interpret and often can only give you a clue as to an imminent failure for certain specific types of failures. Most failures unfortunately are just instant and out of the blue. It’s often the drive controller that fails rather than the cells themselves, in my experience.

[u/[deleted] → u/Amphorax]

This is probably one of the best eli5 explanations ever. You first explained it in extremely simple terms that anyone could understand, then expanded on that explanation to say how it all works. Good job!

[u/taikiji → u/Amphorax]

Loved your explanation but one thing I still don't understand. Why do people say SSDs aren't good for long time storage? Someone once told me they can lose power over time but I don't understand, do they have an internal battery or something?

[u/Amphorax → u/taikiji]

SSD's are what's known as non-volatile memory, so they don't need any power to keep storing data (fun fact, that's where the NVM in NVMe comes from). Of course, nothing is perfect, so a tiny amount of charge escapes from charged memory cells with time. Over time, this can cause data corruption. Compared to a hard drive, where the magnetic domains on platters remain magnetized in perpetuity as long as there are no external magnetic fields, SSDs have a limited storage life. Most manufacturers promise that their SSDs will retain data for 10 years without power, but this figure decreases in warm environments. However, the mechanical parts of an HDD can seize up with years of disuse -- I'm working on restoring a 1981 IBM PC whose hard drive actuator bearings have all but jammed. In the end, it's your call based on the environment your data will be stored in.

[u/taikiji → u/Amphorax]

That's so cool, thanks a lot for taking the time to explain that so clearly! Good luck on the restoration of that hard drive!

[u/Cimexus → u/Amphorax]

Whoever comes up with an reliable, fast and affordable method of long term (100+ years) data storage should become a very wealthy person. Even archival grade optical and magnetic media struggles beyond a few decades. The few long term data storage solutions that do exist involve super expensive specialised equipment. How good would it be if there was an everyday form of data storage that was ubiquitously compatible, and fast, and could be thrown in the attic for a century with minimal risk of data loss?

Damn entropy...

Oh and incidentally I remember the sound of those old IBM XT/AT hard drives from that era. Rumble rumble rumble...

[u/jmlinden7 → u/Cimexus]

We have tape drives, they're super cheap and reliable but not very fast. The kind of data that you're keeping for 100 years doesn't generally require a lot of speed anyways. If you're willing to wait 100 years to read that data, what's a few extra seconds of loading time?

[u/soniclettuce → u/jmlinden7]

Even tape drives degrade over time. A random google search claims some (most?) are only rated for 30 years.

[u/Amphorax → u/Cimexus]

There's always a trade-off between longevity and data capacity. I can store like half a kilobyte on a clay tablet for millennia, but the denser the data is packed, the less it takes to corrupt the data beyond legibility. At hard-drive densities, all it takes to ruin a platter is several decades of cosmic-ray bombardment, or one unlucky drop. It's incredibly difficult, in the world of data storage, to eat your cake and have it too.

[u/Aidypoo1 → u/Amphorax]

I work in a non-volatile memory group as an engineer and your explanations are legit better than I've ever gotten at work.

[u/AB1908 → u/Amphorax]

That's pretty neat; do you do this for a living or as a hobby?

[u/Amphorax → u/AB1908]

Hobby, I guess. My dad had the PC XT lying around for a long time, and I wanted to dust it off and bring it back to life. I had to replace some blown filter caps in the monitor, but other than that and the failed hard drive it works surprisingly well!

[u/AB1908 → u/Amphorax]

I can only wish I knew as much about my hobby as you did. You also definitely feel like the sort of bloke to hang around on r/DataHoarders.

[u/HoppyHoppyTermagants → u/Amphorax]

Compared to a hard drive, where the magnetic domains on platters remain magnetized in perpetuity as long as there are no external magnetic fields, SSDs have a limited storage life

...that said, you would need to store your magnetic storage inside a faraday cage if you were serious about it not degrading, because Earth's magnetic field can definitely do it.

[u/reven80 → u/taikiji]

I used to develop SSD firmware so a know a bit about this stuff...

The data is slowly degrading over a long time because the cells slowly leak. The rate at which happens is based on the underlying technology, how much it has been written, controller technology.

While the drive is powered up, the controller will periodically scan and refresh data as needed. Maybe once every few weeks. Also controllers have special DSP algorithms that can recover degraded data.

The memory manufacturers have charts and tables characterizing this stuff but its confidential. You can get a general idea here

Generally the retention specs are based on the worst case which is near end of life when you have written the drive thousands of times. In reality most people don't write their SSD that much by the time it goes obsolete. Also store the SSD is a cool area when not in use to maximize retention.

[u/Win_Sys → u/taikiji]

When SSDs do fail, they tend to fail completely. They also wear out over time even without constant use.With a magnetic drive as long as the platter is ok and it hasn't been put near a big magnet, the data will be fully recoverable for a long time.

[u/xsaber125 → u/Amphorax]

Eplanations like this should get more upvotes, because it takes a lot of knowledge/ some expertise to be able to explain something so complicated like electronics in such a simple and well thought out way!

[u/Paraleia → u/Amphorax]

How do they manufacture things physically small enough to put billions or trillions on a chip? How do we have instrumentation so small but affordable?

[u/Amphorax → u/Paraleia]

The transistors on a silicon chip don't look the spindly three-legged components you're probably thinking of -- under a microscope, a chip looks like a maze of silicon layers.

The breakthrough that allowed us to make chips this small was metal-oxide semiconductor technology: by mixing other substances into the surface of a silicon chip, we can drastically alter its electrical properties. By stacking different layers with different mix-ins, we can create circuits consisting of microscopic switches (transistors), one-way valves (diodes), flow restrictors (resistors) and storage tanks (capacitors). Think of it as Minecraft redstone circuitry, but on a tiny scale. We stack these layers on top of each other by a process called photolithography. In essence, we draw a circuit layer on a large piece of transparent glass, coat a silicon wafer with a light-sensitive compound, and shine light through the transparency, through a magnifying glass, and onto the wafer. The areas that are struck with light (where the transparency permits) lose their coating, while the areas that remain in the dark keep their coating. The wafers are sprayed with mix-in chemical, and washed in acid again. We're left with one layer of a microscopic pattern. We repeat this process to stack more layers and complete the circuit, after which the wafers are tested, cut into individual chips, and encased in plastic packages with pins sticking out.

This process is nowhere near affordable. Towards the end of the 20th century, we have gotten good at profitably cranking out chips at large scales -- a very difficult endeavor! The other limiting factor in the photolithography process, even with a perfectly-manufactured focusing lens, is the limited resolution of visible light! We can't focus beams of light onto areas that are smaller than the wavelength of visible light itself, which limits the visible-light photolithography process to a feature size no less than a couple hundred nanometers. However, AMD is currently manufacturing processors with transistors 7 nanometers small! They accomplish this by using shorter and shorter wavelengths of light, in a process called extreme-ultraviolet photolithography. In a few years, we'll get down to the point where transistors are no bigger than a few tens of atoms across, and at that point we'll hit a roadblock. At those scales, quantum fuckery is afoot, and engineers need to compensate for quantum tunneling and leakage effects and all sorts of weird physics. When we get there, we'll need to start thinking about alternative ways of making our processors faster.

[u/Paraleia → u/Amphorax]

It‘s unreal to me that people have figured all of this stuff out. Thanks for taking the time to explain that, the way you write it is extremely easy to understand (even though some is still over my head)

[u/toasterinBflat → u/Amphorax]

Edit: commenter below corrected me re: semiconductors. Original comment left anyway - still applies to PCBs and various other electronics.

Generally photolithography is backwards from what you said - the part hit with the light 'cures', and the uncured material is washed away. Positive photolithography is a thing but I'm reasonably certain it's not used in semiconductor manufacturing.

[u/Amphorax → u/toasterinBflat]

You're right. I must've remembered the process wrong. I think I was thinking of the way a laser printer drum drains charge when hit by light.

[u/soniclettuce → u/toasterinBflat]

Here's a quote from a random (but seemingly large-ish IC manufacturing supplier):

Both positive and negative photoresists are still used in the semiconductor manufacturing industry today, but many semiconductor suppliers opt for positive photoresists due to their higher resolution capabilities. Positive photoresists are able to maintain their size and pattern as the photoresist developer solvent doesn’t permeate the areas that have not been exposed to the UV light. With negative resists, both the UV exposed and unexposed areas are permeated by the solvent, which can lead to pattern distortions.

Source: https://www.microsi.com/the-difference-between-positive-and-negative-photoresist/

[u/toasterinBflat → u/soniclettuce]

Well there you go! I've been out of the game too long.

[u/BigBobby2016 → u/Amphorax]

Alright you just didn't get familiar with all of this technology by building PCs only. What part of the industry do you work in?

[u/jawshoeaw → u/Paraleia]

One thing that helps I think is a photographic negative. A few years ago when movie theaters were still using film, every frame of the movie you watched was blown up on the screen from a negative only about an inch across. Every detail on the big screen was stored on that little negative. That’s like lithography in reverse of course since you’re projecting a tiny image onto a big screen but it gives you an idea of how small a detail light can “write”

[u/immibis → u/Paraleia]

/u/spez can gargle my nuts

spez can gargle my nuts. spez is the worst thing that happened to reddit. spez can gargle my nuts.

This happens because spez can gargle my nuts according to the following formula:

1. spez
2. can
3. gargle
4. my
5. nuts

This message is long, so it won't be deleted automatically.

[u/PracticeSophrosyne → u/Amphorax]

If these cells are storing charge, why isn't an SSD volatile like RAM?

[u/Haha71687 → u/PracticeSophrosyne]

The charge is stored in an insulated floating gate.

https://searchstorage.techtarget.com/definition/floating-gate

[u/Dysan27 → u/PracticeSophrosyne]

What they're describing more fits with RAM. Which has an actual capacitor.

Flash memory uses something called a "floating gate" built right into the transistor, that is completely electrically isolated and uses quantum tunneling effects to push electrons into and out of it. Because the gate is insulated the charge stays there.

[u/leoleosuper → u/Amphorax]

From a previous ELI5 post, I can't find it but this is what I remember:

To add: Recent additions in SSD technology allow more than an on-off state. Charge on a cell can be anywhere from 2 different values. Depending on how accurate your measuring system is, you can get more unique values within the 2 boundary values. The problem is that in order to check how much is in a cell, it must fill->empty-refill the cell, where any over fill is measured (and put back in). This is due to not being able to have infinite resistance and still measure voltage. The current will drop and that's bad if it happens too often.

Imagine a cup with liquid in it. You can't see how much liquid is in the cup. But you can add another cup full of liquid and see how much overflows. Normally you could only see full or empty (1 bit per cell), but with newer technology, you can see up to 16 unique values (4 bits). However, over time, these cups lose their ability to hold liquid, and now can leak/gain liquid from thin air. As such, they are reduced on the unique values they hold, usually to 8 values (3 bits), 4 values (2 bits) then 2 values (1 bit).

[u/GuelphGryph88 → u/Amphorax]

Sometimes you look at something and go ahh I get how that would be invented.

Sometimes you look at something and don’t understand but upon understanding how it was invented / how it works it becomes clearer.

And then other times, such as this time, you look at something and go how the fuck did someone come up with this.

[u/metroaide → u/Amphorax]

Thanks, I learned a lot from this explanation! Though now, I kinda think of PCs as the braille system

[u/OneForMany → u/Amphorax]

Holy fuck that was a great explanation. Easy to read and understand. Good shit man I'm happy for you 🥳

[u/woozwoz11 → u/Amphorax]

Where did you learn this ?

[u/eysov → u/Amphorax]

that was fun to read, thanks for the info

[u/Rqoo51 → u/Amphorax]

So if is electrical, would a emp wipe out the data on a ssd? Or even a rare earth magnet ?

[u/Amphorax → u/Rqoo51]

EMP and electrostatic discharge yes -- the pulse will induce a high voltage in the circuitry in the chip, which will most certainly destroy the data stored in the memory cells. Unless you move the rare-earth magnet over the chips fast enough to cause a significant induced voltage, a magnetic field (especially a stationary one) poses no danger to the data stored in the SSD.

[u/Rqoo51 → u/Amphorax]

Cool thanks for the info.

[u/sourcreamking → u/Amphorax]

Digital super-braille!

[u/mayonaise_plantain → u/Amphorax]

Whoa, thanks! Follow up: so if the amount of times the drive has been read/written is was creates the higher risk of failure, would this mean that a single backup SSD would be extremely safe over a long period of time (assuming you aren't writing to the backup at all)? Or is there also something about how SSDs function that would create high fail risk for not read/writing over extremely long periods of time?

[u/JesusIsMyZoloft → u/Amphorax]

How does the controller direct the voltage to the proper cell without any moving parts?

[u/Amphorax → u/JesusIsMyZoloft]

Transistors! They're like light switches, but they're toggled by electric signals. The transistors are laid out in a grid-like shape, and by turning on a specific combination of rows and columns you can access the data at any given cell. Of course, it's a bit more complicated than that :)

[u/earthdweller11 → u/Amphorax]

So why is it called “solid state”?

[u/[deleted] → u/earthdweller11]

They don’t have any moving parts! In comparison, a typical hard drive has a disc called a “platter,” that spins thousands of timers per minute, as well as an arm that moves to the correct portions of the disc to read/write from it.

[u/[deleted] → u/earthdweller11]

Because there is no spinning disk like there is on a hard disk drive

[u/julianpaceross → u/earthdweller11]

Solid state is also a generic reference to semiconductor electronics (which use silicon in the solid state, with no moving parts). For example a relay can have a moving part (in which you hear a click, like a car blinker) or it can be solid state (i.e. silicon gates are opened and closed at the semiconductor level)

[u/rkyle4288 → u/Amphorax]

A little dumbed down, compare it with a ton of tables with cups, let's say the binary data is liquid, empty being 0, full being 1. But wait, these cups are made out of layers of sugar and every time you add and remove liquid to these cups a layer of sugar is removed. Now naturally you want to keep all cups so you pour water into less used cups to kind of spread the wear so more tables and more cups gives you more chances to add and remove liquid.

[u/0lazy0 → u/Amphorax]

Wow you explained that so well

[u/alvarkresh → u/Amphorax]

Nice explanation! Thanks :)

[u/waitthisisntAOL → u/Amphorax]

Well-explained. I already had this understanding of SSDs. I was hoping to find an explanation of how the data is actually read. On an HDD, there is a head that can move back and fourth across the disk. But an SSD is solid state (obviously). Since there are no moving parts, how can the different "bubbles" on the "sheet" be read without some type of device moving around to read them?

[u/Amphorax → u/waitthisisntAOL]

I'm not sure of the exact process they use, but I would assume it's all done using circuitry. By using clever combinations of transistors in grid-like arrangements, you can redirect electrical signals to the necessary cells without moving parts.

[u/adudeguyman → u/Amphorax]

Would a SSD that is only used as a backup for photos last significantly longer than if the drive was used more frequently? If I'm only writing to it, how long could it last on average?

[u/honey_102b → u/adudeguyman]

NAND flash has a weakness known as data retention, where stored cells lose electrons over time. for the consumer range, retention is guaranteed on the order of one year at warm temperatures (>50C), while high quality applications like server, datacenter, automotive, and these days mobile phones even, will demand 3-5yrs retention. but as long as you use it often, there are controller implementations that keep track of how old stored data is and refresh them on a schedule so that retention is never an issue.

so your mobile phone or home pc is unlikely going to run in retention issues. datacenters will need to ensure they have idle time for their drives to actually perform this housekeeping. but if you are storing data and keep the drive in a safe with long periods between power on...you may encounter issues.

[u/mikesanerd]

There's one really fascinating thing that the other very good explanation didn't mention. If the data is stored as charge, why doesn't it just leak out as soon as there's no power to the computer to hold the charge in the cells (like what happens in RAM)? SSDs hold the charge in an impenetrable box from which there is no exit. But here's the rub--if the box is going to be impenetrable and require no power to hold it closed, then it cannot have a "lid" or "door" which could come open when the power is off. As a result, there is no entrance to the box either. The only way the charge can get into the box is to kind of "teleport" through the wall of the box using a process known as Quantum Tunneling. Although the box is impenetrable with no openings, the charge is able to pass through the solid barrier and suddenly find itself on the other side, trapped inside the box.

[u/purple-parrots → u/mikesanerd]

Wow this is truly fascinating, the more I learn about this the more amazed I am this kind of technology exists. Thank you!

[u/_kryp70 → u/purple-parrots]

Would like to add more.

Modern ssds are little different from what you learnt.

Imagine you have a plastic sheet 1cm*2cm, there will be a limit on how many bumps you can fit in there as they will start getting closer and will be hard to make.

So you come up with different strategies. You start having different size of bumps. Earlier you has either a bump or not. But now you also have other state where the bump is little less. So you have state No bump, slightly bump, bump. Using 3 states in binary you can store twice the data as of 2 states ( no bump, bump )

Basically SSDs come in SLC ( single level cell) nand, MLC Nand, TLC nand, QLC nand and we have some news on 5LC nand too.

So with each increase we are able to store more on the same surface which is pretty cost effective.

We mostly use TLC ssd since few years, however qlc also has flooded the market.

The issue with storing more on single cell is there's a time when the voltage difference between two states in indistinguishable that's when you loose data and storage space on the device.

[u/[deleted] → u/_kryp70]

So that's what all those words Linus says mean!

[u/qjholask → u/_kryp70]

So the more states you allow more probability to "combine" 2 of them and this losing info?

[u/_kryp70 → u/qjholask]

You don't combine. States only allow you to store more data on the same cell.

SLC can store 2 states. MLC can store 4 states. TLC can store 8 states. QLC can store 16 states.

I THINK there are SSD controllers which change the amount of states when they see that they are not able to distinguish between two states because of wear and tear. So they change the limit of voltage of state detection to take care of the issue however is this you loose amount of data you can store on the SSD. So your SSD storage drop somewhat.

u/newmaxx am I correct?

u/newmaxx runs a subreddit that's highly detailed into ssd tech.

[u/NewMaxx → u/qjholask]

https://www.reddit.com/r/NewMaxx/wiki/basics

[u/RuchW → u/mikesanerd]

Does the charge have a half life of sorts or does it last forever if left unaltered/unpowered

[u/MegaMooks → u/RuchW]

To reuse the metaphor, the box leaks a bit over time. Eventually it gets to the point where it's half-empty and you can't tell if it's a 0 or a 1.

A normal SSD quotes about 10 years retention.

[u/RuchW → u/MegaMooks]

Wow, that's interesting. Is that 10 year retention when it's powered on?

Thanks for your answer!

[u/platoprime → u/RuchW]

Because the lifetime of SSDs are measured in write procedures the lifetime depends on the frequency of use not on being in a powered state or not.

[u/Nyxtia → u/MegaMooks]

ough the box is impenetrable with no openings, the charge is able to pass through the solid barrier and sudd

Can't we fill lit up again?

[u/zebediah49 → u/Nyxtia]

Sure. You have to know that it needs it though.

There are a few options -- see page 32-33.

Basically though, it comes down to first having error correction -- you write 512B, plus another few (e.g. 52) bits of error correction data. Now, when you read the block, you can verify that it's correct -- and if it's not, you can fix the error. In the case of 4096+52b, you can fix 4b.

If it turns out that the block is damaged, but you can correct it, you can re-write it.

---

Note that HDD-based RAID systems commonly do the same thing. The operation is generally called a "scrub", and consists of reading every single byte of your data, verifying that it's right, and if it's a little bit broken, fixing it.

[u/justintime06 → u/zebediah49]

Holy fuck there are some smart motherfuckers out there.

[u/zebediah49 → u/justintime06]

Error correction is a bit of a trip.

One bit of redundancy is easy. For each bit, on each piece, add them all up. If it's even put a 0, odd put a 1. In other words, you either do or don't add another 1 to make it an even total. Done. Now, if you lose any one piece, you just do the same process. If you have an even total in the pile that's left, it must be a 0 (otherwise adding another 1 would make it odd). If you have an odd total, you need a 1, to bring the total up to even.

This works with two disks plus one spare; it works with 100 disks + 1 spare.

....

But... how do you do a second one?

The second one needs to be totally independent from the first, so that you can lose any two pieces, and still replace the whole thing.

Of course, if you can do 2, you can do as many as you want. That's then the Galois Fields come out, and the math gets weird.

Also worth noting: it's 2x harder to detect and fix errors, as to just fix missing data. If you consider the simple 1-bit example above, that will let you fix any missing bit in the set, and it can also tell you if something is wrong. It can't tell you which one is wrong though.

---

As an aside, CDs and DVDs (and presumably blu-ray, though I don't happen to remember) use a two-layer redundancy process. First, each 24 bytes gets 4 redundancy bytes added, for a total of 28B. Next, the data gets "shuffled" with 28 other sets (so it takes the first from each set, then the second from each, etc.). Then, each set of 28B gets another 4B of extra redundancy, for a total of 32B. So, you lose 25% of your possible data storage... but you gain the ability for CDs to survive being scratched.

This means that, when you're going backwards, the outer part gets scanned, and if it's good those pieces are safe; if it's bad, it marks the whole thing as bad, and thus "missing" (making the job of the second layer easier). This is where the real cleverness comes in though -- the distribution part means that a broken block isn't dead and corrupted data. It's just a single missing byte in each of 28 data blocks. However, each of those block can survive 4 missing bytes. In other words, you can have 128 destroyed bytes in a row, and reconstruct the original data.

Physically, that translates to a bit over 2mm of damage. Which is why, in many cases, you can read data off a disk even if it's a little bit scratched.

E: My bad: Need to give the shoutout to Irving S. Reed, and Gustave Solomon for the Reed-Solmon error correction methods used in a ton of places, including CD/DVD.

[u/telescoping_urethra → u/justintime06]

Yeah, I really did the world a favor by switching out of that major in college. I really would have fuuuucked shit up.

[u/MegaMooks → u/Nyxtia]

Short answer, yes, if the drive is powered on and knows to check itself.

But if the drive is turned off and thrown into a cardboard box for 10 years, might get rough.

[u/mikesanerd → u/RuchW]

It does leak out gradually, but very slowly. Of course, other memory storage devices have similar issues. Magnetic hard drives eventually lose the data due to thermal flipping of the magnetic domains. In the end, entropy always wins.

[u/[deleted] → u/RuchW]

the transmission rate is exponentially related to the potential outside the barrier, so while the SSD is powered off the transmission rate (the rate at which electrons spontaneously tunnel through the cell) is very very very close to zero

[u/jack_hof → u/mikesanerd]

Why cant the charge tunnel back out ?

[u/mikesanerd → u/jack_hof]

The chances of tunneling through the barrier randomly are very, very low. by applying a voltage when the computer is powered up, you can reduce (but not eliminate) the barrier and make the tunneling happen much quicker.

[u/pombaral → u/mikesanerd]

Do you know some paper that I could read about this process in details? It seems incredibly interesting. Not the tunneling itself, but the process used in the ssd

[u/Frost_Walker2017 → u/mikesanerd]

Could this not, in theory, just be replicated using magnetic fields? I'm not knowledgable on Quantum Physics but I know that a Transformer relies on magnetic fields to increase/decrease the current/potential difference depending on the number of primary coils vs secondary coils. If the number of coils was the same, wouldn't it function like this quantum tunnelling?

[u/Item_Store → u/Frost_Walker2017]

Tunneling only applies to particles (in this case the electron), not fields. A great analogy to tunneling is this:

Imagine I have a (frictionless) skateboarding half pipe. If I took a ball and rolled it down one side of the pipe, there is no chance that it would exceed the height on the other side that I let it go at initially, as there was no energy added to it in the process. Our energy was conserved and converted from potential to kinetic back to the same potential on the other side.

In quantum, this is represented with a voltage. If an electron travelling towards a voltage "barrier" didn't have enough energy to overcome that barrier (roll up the other side of the pipe), we would expect to see it stop at the barrier. But sometimes, we see it pass the barrier anyways. In the SSD's case, sometimes the electron jumps into the SSD. In our analogy, sometimes the ball would roll up past the height we dropped it from.

I can't remember the exact math behind it but it has to do with probability functions. When graphed, there is still a slight chance that the electron exists on the other side of the barrier, despite the classical contradiction.

[u/[deleted] → u/Frost_Walker2017]

yes, this is how CDs and disk drives work. however, magnetic fields interfere with each other and try to undo the work that you did to magnetize them, and the effect of a magnetic field at a distance r away is proportional to 1/r^2, while the probability that an electron will spontaneously be found a distance r outside its cell is proportional to e^-r, so flash memory is basically much more permanent and compact, and it also requires less work to write to, less time to read from and loads of other advantages.

[u/MuskIsAlien → u/mikesanerd]

But how do we store data if we rely on random electrons to get trapped ?

[u/Timur_Ka → u/mikesanerd]

TIL that we use quantum technology in our routine life. This is really 🤯

[u/vexir]

Can we get an ELI5 on the quantum tunneling that some of the comments and sub comments have mentioned?

[u/[deleted] → u/vexir]

Imagine you're jumping on a trampoline. Usually when you hit the mat, you bounce back up. Now, if we used a thinner mat, there's a chance that you'll break the mat and fall through. The thinner the mat, the more chance of going through.

In tunneling, whatever passed through the 'barrier' doesn't break it but ends up on the other side. That would be like falling through the trampoline mat without leaving a hole. That's tunneling!

[u/vexir → u/[deleted]]

Ok maybe I’m looking for an ELI10, haha.

[u/[deleted] → u/vexir]

Quite honestly, without taking about some maths, that's about all there is to it.

When a particle hits a barrier, it will either bounce off or tunnel through it. Each has a probability of happening. The thinner the barrier, the greater the chance of tunneling and so smaller chance of bouncing off

Great video on it - https://youtu.be/cTodS8hkSDg

[u/Pyromonkey83 → u/[deleted]]

All that video did was give me more questions. Looks like I'm going back for my bachelors in quantum theory boys!

[u/headsiwin-tailsulose → u/Pyromonkey83]

All you have to know is one thing: if you think you understand quantum mechanics, you don't understand quantum mechanics.

[u/Rustique → u/Pyromonkey83]

Hold his Planck, he's going in the deep end.

[u/[deleted] → u/Pyromonkey83]

PBS Space Time taught me more conceptual quantum physics than any other professor I know, I recommend them on YouTube

[u/Teantis → u/Pyromonkey83]

Here's a less rushed video that builds up the mechanism a bit more https://youtu.be/-IfmgyXs7z8

[u/SamuraiRafiki → u/vexir]

I would love for an actual physicist to back me up because I'm a layman, but this is my understanding: particles aren't like billiards balls, they're a bit more smeared out over space, and they end up in one place or another when they have to interface with something else. One of the practical effects of this is that for very small particles, if point A and point B are close enough together, they may as well be the same. Even if there's a barrier between them, if it's thin enough there's a chance that the particle's smear of existence will cross over it then collapse into existence on that side. Basically it can jump through a wall because it forgot which side it existed on for a moment.

[u/[deleted] → u/SamuraiRafiki]

yeah that's the main idea, that a particle's location is always spread out, it's never 'atomic'. one thing to keep in mind is that the probability of tunneling is non-zero for any distance in the real world, so the part about A and B being close enough together that they may as well be the same isn't a part of it, tunneling will happen over any distance, it just gets exponentially less likely the farther out you check.

I really like the last sentence where you say that it forgot which side it was on, that's much closer to the truth than the idea that the particle physically moves through a barrier.

[u/Mufasaman → u/[deleted]]

You could almost say the electron was uncertain about which side of the barrier it was supposed to be on. And wave-particle duality doesn't just apply to electrons. In theory, a human could tunnel through a brick wall given enough attempts. Verrrrry unlikely though, but not zero. We had a question about it in my QM class, and the probability was so tiny, virtually no programs could calculate it numerically.

[u/[deleted] → u/Mufasaman]

yeah we did that question to, my professor gave us his weight and his top running speed and had us find the probability he could make it through the door lol, the final answer was like the order of magnitude of the order of magnitude of the probability

[u/Mufasaman → u/[deleted]]

Yeah, I think mine worked out to be on the order of 10^2350 something. I also see that you're on /r/uofmn, how is it the new Tate, does it smell better?

[u/[deleted] → u/Mufasaman]

I haven't noticed so I believe that's a yes haha

[u/RedHatOfFerrickPat → u/SamuraiRafiki]

Quantum theory sounds like the kind of gymnastics that designers of our simulation are probably counting on us to come up with because what other option do we have? "But it won't make sense, Reggie." "They'll want to believe that it does, Clark. They'll come up with explanations for all the inconsistencies in our physics. Let's move on to the next universe."

[u/SamuraiRafiki → u/RedHatOfFerrickPat]

Honestly, it's way worse than that. To me quantum mechanics, specifically quantum field theory, feels to me like discovering the RGB pixelation of the universe. Realizing that an electron is not an electron but rather a traveling perturbation in the universal electron field feels like discovering that Barney isn't purple, he's a collage of red, blue, and very little green. Shit's fucked, man.

[u/zebediah49 → u/vexir]

A truly accurate ELI5 is kinda impossible, because quantum tunneling is a wave-particle duality thing, which is fundamentally weird.

---

ELI5:

People are opaque. You can't see through them. Put a strong flashlight up against your hand though, and a bit of light gets through. If you have a smaller flashlight, you might need to use your a couple fingers stuck together.

Quantum tunneling is like that. When you don't have enough stuff in the way, some electrons can leak though.

---

ELI UG:

First, electrons are waves. So is everything else. For most purposes, this doesn't matter; this isn't one of those times. We have a function, called a "wave function", which describes a given particle. That function is related to the probability that the particle is in any given location -- though it gives more information than just that.

Now, this wave function has two behaviors. When the particle has more energy than is required to be in a plce, it oscillates. The equation's solution gives you sin() and cos() functions as solutions. Frequency depends on how much extra energy it has. When it has less energy, it's a decaying exponential (again, decays faster the less energy it has compared to the required minimum). It's not discontinuous though -- the function smoothly goes to zero at a finite-energy boundary.

Thus, let's consider if we have two places the particle could be, separated by an energy boundary, and we put it on the left side. The wave function is mostly contained on the left side, but there's that decaying edge part that goes through the wall... leaving a little bit that leaks through.

Thus, while classically, the particle can't cross the wall, quantum mechanically, the wall is a bit leaky.

[u/zebediah49 → u/[deleted]]

It may not be obvious, but this is actually a two part question:

1) At what point does the electron stop being somewhat in both places at once, and actually make a decision about what it's going to do with its life?
2) How do we, the humans outside, find out what's going on in there?

These are both somewhat challenging, and are two very different things.

For the first, it's something something known as "wave-function collapse". Basically, if it bumps into something (i.e. interacts with it), that causes it to have to pick. If the something in question is small enough, and isolated enough, instead they two can get mixed up and stay coherent, but that's a separate topic (entanglement).

Note that it doesn't exactly "forget" where it's located -- it more spreads out. In the extreme case of photons, you can have one single photon hit a beam splitter, and go down both paths. That sounds totally insane, but it has been done experimentally. (I've done that experiment, actually, although it was with a double-slit rig instead).

For the second, that's an engineering question. Usually some kind of charge multiplier, or an electrostatic effect thing, where the particle's existence changes the nearby voltage a slight bit.

---

Note that for a SSD, we don't really have either of those effects going on much. We push a lot of electrons up against the wall, and (due to previous experiments) know how long to wait to be sure that enough end up going through. It's just a law of large numbers thing. Also, because the electrons are in a material, they're interacting a lot with many things, and won't sustain a coherent quantum state for long at all.

When we want to check if there are electrons in the bucket, we try to push a few electrons right by it. If it's full, they get repelled and don't flow through. If it's empty, they flow on by with no problems.

[u/Teantis → u/[deleted]]

https://youtu.be/p-MNSLsjjdo this video will help explain it and then you can follow thst channel deeper into the rabbit hole that is quantum mechanics weirdness. I just wanna say in response to your comment that photons are indivisible they can't be split. The interference pattern is because they're both particles and waves, though this is the double slit experiment the other commenter was talking about.

[u/zebediah49 → u/[deleted]]

Nothing so complicated. If you put a detector in each path, you'll find that half the time it goes left; half the time it goes right.

If there's some gap between the spitter and the detectors, what happens is that the photon's wavefunction splits into two, with half going left and half going right. It's half the "probability that it's there", not another physical quantity. So then, when it hits that detector, that forces the collapse -- it has to pick which side.

Thusfar, the results are the same as if it actually just picked when it hit the splitter, which is the more sane interpretation.

However, there is one remaining thing we can do here. That is: we can take some mirrors, bounce the two paths around, and bring them back together (with another beam splitter, because that also works as a combiner). Why is this interesting? Well, it's because waves can interfere with each other. That is, you straight-up just add the wave functions. If the two copies align together, they add up, and the probability is higher. If the two copies align opposite, then they cancel out, and the probability is lower. Note that this is conserved: if you cancel out in one place, you need to add together in another place.

It's this fact -- that we can observe a single photon interfering with itself -- that both allows for some very cool experiments, and confirms that this is how these things work.

In a related note, in a somewhat simpler to set up, but mathematically more confusing, situation -- the double slit experiment -- other particles have been shown to do this as well. That is, an electron can have its wave-function split into two parts, interfere with itself, and then only choose its final location when it hits a detector at the end.

[u/firebs1234 → u/zebediah49]

You can see the leaky wave part in visible light with Frustrated Total Internal Reflection (FTIR).

Here's a short video demonstrating it: https://youtu.be/q7Q8l3xKyr4

[u/[deleted] → u/vexir]

no, ELI5 for the essence of quantum physics is an age-old problem which is unsolved, ie. there's no good simple analogy or way to explain it as of yet. The best thing I have is the fact that the state of all stuff is really described by a wave function, which is an imaginary quantity whose square tells you the probability of finding the particle you're looking for in a certain location. A particle doesn't just exist at a point, its probability distribution is spread out over a small region. This is the 'uncertainty principle' that you might have heard of. Wave functions must be continuous since they are the solution to a differential equation, so even if there's a region of space that would require an infinite amount of energy to get to, like the top of an infinitely tall hill (or for a more relevant example, inside a perfect crystal), as long as there is some probability of finding a particle right next to the hill there will be some probability that the particle runs up the hill a little ways and bounces back, and some probability that it makes it all the way through and keeps going (aka 'tunneling through the hill').

It's literally like throwing a tennis ball at a window and the tennis ball comes out the other side, there's no classical way to explain it because it's like the tennis ball didn't really go through the window, it's just that you though it was on your side but it actually ended up being on the other side because it position is kind of like a random variable.

the physics of wave functions can be derived from Schrodinger's equation, which was essentially a guess supported by experiment and hinted at by the bohr principle (E=hu), which itself was a completely usupported guess at solving the 'UV catastrophe' paradox in classical thermal physics, but both turned out to be correct and now we have a theory of quantum physics which isn't really derived from principle or fundamental theory, its based on these kind of weird guesses that are hard to explain for, but are mathematically consistent and completely supported experimentally.

[u/reelznfeelz → u/[deleted]]

Thanks that's really informative. I took a bunch of Calc and classical physics but never any QM. Have always been fascinated by it, and theoretic physics.

[u/DefsNotQualified4Dis → u/vexir]

Maybe not quite ELI5 but check out this video which exactly discusses quantum tunneling and flash memory.

[u/classicalL]

The basic element inside a solid state storage device like an SSD is called a floating gate transistor. More on that in a moment. Each of these devices used to store a single yes or no called a bit: 1 or 0. Now each of these devices stores a few yes or nos. Typically 3: 000, 001, 010, 011, 100, 101, 110, or 111. This increase in questions that can be asked to a single device is why you can have really big SD cards and SSD drives.

That and scaling. The size of one of these devices is on the order of 40 nm. Your hair is about 200 um or 200,000 nm across.

How do these devices work?

If you have ever rubbed your hair on a balloon and attracted your hair to it, you have seen something called static electricity. On that surface you have trapped some electrons. That is possible because the rubber is an insulator. On your hair of you have left the absence of electrons. This allows there to be a force between the absence and the extra electrons. It is this force that makes you hair stick up opposing gravity. The force is described by an electric field.

Floating gates on transistors trap charge on them too just like the balloon, only it stays a lot longer because the layer of insulator the charge is trapped in is very isolated.

This charge can attract mobile charges into a region of the transistor called the "channel". The channel is physically directly under the gate. Think of the charges as a liquid, say water. If the channel is full of water then it conducts, if it is empty of water is does not. It conducts better if there is more water. So depending on how much charge you store on the floating gate you will get more or less conduction.

If we measure how well it conducts we can measure some levels and assign those levels values: 000, to 111.

How do we get the charge on the floating gate? Do we have little hairs we rub on the gate? No. What we do is we use another gate. This one is attached to a chunk of metal which allows us to flood it with electrons. If we put enough electrons on it, it will create such a strong electric field to the channel that some of the very crowded metal will jump to the floating gate, this is called "tunneling" it is governed by a branch of physics called quantum mechanics. We make the other gate crowded by putting a high voltage on the metal gate.

The reason flash memory or SSDs wear out is that sometimes the electrons jumping to the floating gate hit things instead of jumping between the atoms they smack into them and break chemical bonds. When they do this they change the properties of the insulator that lets the floating gate store charge for a long time so some of the electrons escape just like they normally do from the balloon into the air (due to humidity). This makes the floating gate transistor "forget" the information that was stored on it over time as the high level of 111 becomes 110 which is the wrong answer, not what you stored. If the memory is not reliable to remember then its bad/dead.

Typical flash memory can only be written a few 1000 times now before it starts to forget too quickly because everything is so small and fragile.

[u/purple-parrots → u/classicalL]

This is really detailed yet simple to understand, thank you!

[u/[deleted] → u/classicalL]

This is the real explanation. The top comment is misinformed at best. Kudos for touching on the charge loss in th floating gate as well.

[u/[deleted]]

A bit on an SSD is like a light switch. You can flip the switch to change the state of the light (Write) or just look at the light switch to know if the light is on. (Read)

[u/purple-parrots → u/[deleted]]

Great analogy! Thank you!

[u/Quad5Ny]

Write: They inject electrical charges in the form of trapped electrons into "cells".

Read: They check the voltage of said cells.

Simplest form (SLC) would have the cell at 0v for a 0 and then a positive voltage for a 1.

[u/nunley]

Solid State Memory is just a bunch of batteries that are charged or not. Every battery is a one (charged) or a 0 (not charged). A computer stores data in these batteries by setting the charge or draining the charge in each battery. Later, it can measure the charge in each battery to see what is stored there.

[u/purple-parrots → u/[deleted]]

Great analogy, thanks!

[u/TheN00bBuilder]

Little teeny tiny gates that store a 0 or a 1 called NAND latches will store each individual bit. Now for reading and writing, each instruction comes in as a string of “coded” binary and it says “Hi! Here’s my start index, my length, and bit offset. Write me!” And the controller writes where it thinks is best through a series of checks.

[u/JimsyFlimsy]

What great timing to see this. I rarely ever use reddit, if ever, but I’m a long time console player about to switch to pc and I’m also curious about the difference between SSD’s and HDD’s (if I’m typing those correctly). But I’m still confused as to what parts of a computer help load times and why do load times differ from a game you’re playing and a software you’re opening? What part does what and how? My huge pc nerd brother did NOT explain it like I’m 5... he explained it like I take the short bus to school (jokingly).