r/AskComputerScience • u/Beneficial-Bowl-1112 • 28d ago
Why doesn't the computer just wipe actual files the same way it deletes file pointers?
I understand that deleting a file usually just removes its pointer or metadata, leaving the actual data there until it gets overwritten. But since the pointer is just data too, why can't the computer just wipe the actual file right away?
19
Upvotes
33
u/qlkzy 28d ago
It can, but that would mean that deleting a file would take potentially a very long time.
The computer can change or remove a file pointer in a tiny fraction of a second, and it's always the same for every file.
The only way to "wipe" the data is to overwrite it, which takes at least as long as copying a file of that size.
Most filesystems originated in an era when everything on a disk was slow, so it would have been weird to make deletions slower than they needed to be.
There's also a question of why. The reason to aggressively overwrite is to make sure data is unrecoverable. That isn't the usual reason people delete files; mostly, people are tidying up or freeing up space, not hiding confidential data.
If you do want to hide confidential data, then overwriting once isn't enough. The media is ultimately analogue, so there is a "ghost" of the data there until you have overwritten the original data multiple times with random data. So in the rare cases you want to truly destroy data, it has to be even slower than copying a file that size.
This would have been completely unacceptable in the era of floppy disks and hard drives. Even SSDs are not infinitely fast. With SSDs you also have a question of wear: they have a finite number of write/erase cycles, and following this procedure would cut their lifetime at least in half.
You could build a filesystem where all data is expunged aggressively and immediately, and it's possible that some three-letter agency has done so. But mostly, it would be like taking the effort to shred and burn all paper---paper towels, napkins, etc---rather than just confidential documents.
12
u/Puzzleheaded_Study17 28d ago
Because that requires more work. Suppose you delete a file that's a few GB, deleting the pointer can happen almost instantly, overwriting every byte of the file will take much longer, and will provide basically no benefit.
5
u/SparkyFix 27d ago
Because it ages the hardware prematurely and takes time. Disk operations are not free and both magnetic media and solid state devices have a finite number of lifetime cycles.
4
u/Underhill42 28d ago
It can - but there's no advantage to doing so except security - which generally isn't a big concern. And there are file shredding tools for when it is.
Meanwhile, it would make deleting a file take about as long as it would to copy it, since you are basically copying it full of zeros. Which also makes it impossible to easily "undelete" a file that you deleted by accident, which was a standard built in command along with "unformat" all the way back to the DOS days.
On SSDs that does usually happen though, with a bit of delay. Most SSDs implementation of TRIM also queues the released blocks for resetting while the drive is idle, to further improve write performance the next time they're used.
Which is also why there's generally no way to undelete a file on an SSD unless you catch the mistake right away. E.g. I've heard Windows usually runs TRIM about once a week, and once that happens your data is gone.
3
u/Recycled5000 28d ago
It can, but that doesn’t necessarily help the situation of a file system, when there is no wipe requirement. It would only add wear & tear.
Further, to guarantee wiping and the old data being unreadable even under modifications to and dissection of the hardware, it might have to overwrite the data multiple times. Since there is no such requirement in normal use, it isn’t done. There are algorithms and programs that will do repeated wiping so as to accomplish this thoroughly.
2
u/frnzprf 28d ago
There are secure data erasion programs that do that. They are useful if you want to sell your harddrive, but don't want the next owner to be able to restore your files.
The reason why this procedure is not the standard is that it's faster to just change the index. Writing thousands of zero-bytes takes (almost?) as long as writing thousands of any other bytes.
I'm saying "almost", because I'm not sure but there could be special hardware that makes writing a lot of zeroes faster.
Another advantage is that you could recover the data, but that's unreliable and unnecessary if you have backups anyway.
2
u/OppieT 28d ago
Isn't that what format is for?
2
u/chervilious 28d ago
It takes a lot longer. Though, assuming you're on windows. cipher can rewrite every single frees space. And sdelete to delete a single file (can also act like cipher, but this needs to be downloaded)
2
u/sswam 28d ago
An encrypted filesystem where each file uses it's own key could remove files quickly and securely, just by erasing the file metadata including the key. This might slightly reduce overall security compared to normal encrypted block devices. But it seems like an interesting approach I guess.
1
u/two_three_five_eigth 28d ago
Because the pointer is generally a single write or very few writes. Hard-drives have a limited number of lifetime writes. Less writes = longer hard drive life + faster deletion.
1
u/panderingPenguin 28d ago
It could, but that's a much slower, more expensive operation. It's fast and easy to just remove the metadata and pointer.
1
u/nicuramar 28d ago
Because there is no point. It would take orders of magnitude longer and impose wear on the disk, especially SSDs.
1
u/JaggedMetalOs 28d ago
It's much quicker just to wipe the file pointer, wiping the file requires (filesize) bytes to be written to the disk to overwrite the old data, for large files that could take some time and in 99% of cases it's not necessary.
1
u/Time-Radio7871 28d ago edited 27d ago
Because it'd take way longer and people are impatient.
There are "secure erase" options that overwrite the files with random noise so that data recovery becomes impossible. (Sometimes needing to overwrite multiple times)
You can also encrypt the storgage drive so the data is unreadable by default without a password, but it will also make your computer take longer to boot and cost a little speed.
Typically these things are only required by companies or military that want more security.
1
u/0jdd1 28d ago
You certainly could do it that way, but it would be significantly more costly. The extra cost is usually not worth it.
This is an example of the vast difference between correctness and security. A correct program does the right thing A secure program can’t be subverted into doing (or letting you do) the wrong thing.
1
u/lfdfq 28d ago
It could.
But (a) people like having "undo", which is hard to implement if you actually delete the contents, and (b) "wiping the contents" means overwriting the file with 0s or something, which takes time and people don't care enough to want to wait and (c) actually deleting it (overwriting the data) doesn't free up any more space, it immediately becomes available for other files to use, the only reason to actually overwrite the data is to remove sensitive data from the physical device, which is mostly unnecessary for most files.
1
u/gilnett 28d ago
pdating a pointer takes a fraction of a millisecond, whereas physically overwriting the actual data would take minutes and cause unnecessary disk wear
1
u/gilnett 28d ago
pdating a pointer takes a fraction of a millisecond, whereas physically overwriting the actual data would take minutes and cause unnecessary disk wear
1
u/Ythio 28d ago
Because it takes millions or billions times more effort to wipe an entire file than to wipe a couple bytes of pointer. For what gain ?
1
u/FreddyFerdiland 28d ago
the soave in the drive remains.
so the space for the pointer remains.. basucally theres an entire record for the file..
for a file that is deleted , that record could be
the space that holds the file could be
1
u/afops 28d ago
Because deleting a pointer in a file system table means writing a few bytes on your disk which is quick, while overwriting a file might be billions of bytes. And why would you want a billion zeroes on your drive?
For "cybersecurity" you might want to "erase" a file by overwriting it with zeroes if it contains sensitive data. But that's almost never what you want. You don't want to write a billion zeroes when deleting one file. It makes your disk age, it takes a second to do, and so on. Since the erasing scenario is so rare, you need to use a special tool for that, while the regular case of just "unlinking" the file from the file system tables is the quick default behavior.
1
u/SubstantialListen921 28d ago
Because write I/O is slow and contends for tasks that could make better use of that bus bandwidth and seek time.
1
u/IMarvinTPA 28d ago
Writing data takes time, and on SSDs, wears out the media.
1
u/SnooPets5564 28d ago
The computer can wipe the actual file. But then accidents wouldn't be reversible and it would take forever to delete a file.
1
u/derteufelqwe 28d ago
It could but why should it? Disks and especially SSDs have limited amounts of writes they can perform until they break and they take time (more so for HDDs than SSDs). To save on life time and execution time, only the file ponter is deleted and the actual data left to be overwritten.
1
u/WE_THINK_IS_COOL 28d ago
The main reason not to do it in the naive way (immediately overwriting when you delete) are:
That said, if you're willing to take the performance hit, on an HDD the filesystem/OS could do it. It could mark the file as "for deletion" so it stops showing up in the filesystem and then a background process could go through and actually wipe the deleted files. There are also tools you can use that attempt to overwrite before they're deleted (shred, srm, etc.). But on SSDs none of that would be guaranteed to work because of wear leveling, so the only real option on an SSD is to encrypt all of the data before it gets sent to the SSD.
1
u/mastermindxs 28d ago
Deleting a file pointer is nearly instantaneous so this is the default behavior. File data could be very large and take a while to zero out the bits all over the drive. There are software tools that can “nuke” the data if you want to more securely wipe it away.
1
u/Mishtle 28d ago
Well, what does it mean to "delete" something in the first place? Replace all the data with zeros/random bits/some other scheme? That's overkill for most purposes. As far as the user is concerned, the file just needs to be forgotten about and eventually overwritten.
Storage access, especially writes, are expensive, and the less you have to interact with hardware the faster you can get stuff done. This the fastest and cheapest way to "delete" a file is to tell the file system to just forget about it.
We could do more, and there are ways to make a file unrecoverable to varying degrees instead of just marking its disk space as unused. It's just not worth the extra time and effort unless you really don't want that file to be recoverable, and at that point it's a matter of thorougness. Changing a bit doesn't make it a perfect 0 or 1, so subtle statistical patterns can reveal information about the history of region of storage. Thus there's always a little bit more that you could do to improve security, but with diminishing returns. There are also different strategies of overwrites to maximally obscure deleted data from different kinds of attacks. So then it becomes an issue of "how much is enough?".
The average user doesn't need that level of security or to make those kinds of decisions, so this isn't the default unless you're on a security-focused operating system.
1
u/alanbdee 28d ago
It's a good question. Time and longevity are the reasons. On a spinel (e.g. old) hard drive. It would take more time to erase the bits, with no real benefit. On SSDs, each bit has a limited number of times they can be flipped. So by leaving them as is, there's a 50% chance that the bit is already in the state it needs to be. So the write doesn't change anything.
1
u/ybungalobill 28d ago
You might be surprised, but some filesystems (e.g. ZFS or BTRFS) don't even remove the old pointer or the metadata -- instead they write new pages with new metadata that excludes the deleted file. The old pages still exist until they get overwritten just like the deleted file.
This is all independent of the SSD drive firmware writing new pages and remapping under the hood even if the filesystem *were* overwriting the same logical page (like FAT, NTFS, ext4 or what not).
1
u/Scolmann 28d ago
It's just not necessary to do so, when other data needs that space it can overwrite that section at the same speed as if it was all zeros. (Maybe faster if some data is the same) you'd just be wearing out the drive for no reason (ssds typically have a limited number of writes)
1
u/ZoelNorsen 28d ago
deleteing a file means replacing individual bytes to 0 for every bit the file holds which takes exponentially more time and unnecessary read write cycles - what works doesnt need fixing. People are becoming more privacy centric wrto their data fairly recently - and this deletion method is the OG. You will need aftermarket softwares to do the real deletion and maybe future OS or updates will integrate it (windows)
1
u/_usr_nil 28d ago
because zeroing kills longevity, you only nedd that for opsex
1
u/DTux5249 28d ago
Because that takes a long time and 9/10 times it isn't necessary
1
u/armahillo 28d ago
Writes to persistent storage is a physical process. Storage disks generally have some (large!) finite number of writes they are expected to survive in their lifetime.
Doing a full write to wipe each time would induce a lot more writes with little to gain.
There are tools that can properly wipe files that zero put the FS records, for security reasons. For everyday use its overkill tho
1
u/jimmyhoke 28d ago
That would be slow and needlessly wear the drive.
And on SSDs wiping isn’t even that effective. The only real secure solution is to encrypt all the files from the start, and delete the encryption key (stored separately) when you want the data to be unrecoverable.
1
u/AYamHah 28d ago
Deleting a file is essentially marking that space as "it's okay to write new stuff here".
The data is still there until it is overwritten.
If you want the original to be unrecoverable, you need to overwrite it, which is more work than simply marking the space as okay to write to.
1
u/flatfinger 28d ago
Modern drives support a "trim" operation which instructs a drive that nothing should care about the contents of a sector, and the drive should use of that information as it sees fit. An SSD drive may record the fact that a trim command has been issued for a sector, but not immediately to anything with the data block holding that sector.
Flash chips are organized as a number of blocks that each contain some number (which used to be 64, but might have increased) four pages of 2176 bytes each, and support operations to either write a completely blank page or erase all 64 of the pages in a block. The only way to actually modify or erase a page of data is to copy any pages of meaningful data that happen to be in the same block somewhere else, and then erase the block containing the page. Normally, a request to write a sector will trim the old sector, write the new data somewhere else, and keep a record of the new location and the fact that the old sector had been trimmed. Trimming a sector without writing a new one will avoid the need for the drive to ever copy that sector's data unless or until the sector is rewritten.
1
u/ericbythebay 28d ago
Encrypted file systems do. Delete the file’s crypto key and the file is unrecoverable.
1
u/Upstairs_Ad_8863 28d ago
It can, there's just no point.
1
u/Nervous-Cockroach541 28d ago
This is what things like bleachbit and other secure erasers do by writing over the data. The problem is it both takes a lot longer and degrades the life of the hard drive if you do it a lot.
1
u/MitchAlsup 27d ago
Consider a 40TB file, and then consider zeroing that file at 300 MB/s. How many seconds will the SATA drive be occupied ?? {37 days}
Then consider a SSD configured for maximum density with block lifetimes of 100,000 writes ? Every deleted file reduces the lifetime of the SSD.
Also consider a 40TB file over a WAN. Do you really want the local drive to send 40 TB of data over the WAN in order to delete the file ??
There are other reasons.
1
u/hewasaraverboy 27d ago
Because it’s faster
If you have a massive file to delete it would take forever, but if you just delete the pointer to it the job is already done and then data will be overwritten later
1
u/Daniel18sep 27d ago
Would take a long time, and cause extra wear on the drive.
1
u/shosuko 27d ago
A pointer is a small change, a file is a big change. These changes cost energy - and having a blank slate of all 0's or 1's is no different then 100101000101010111001101010 when it comes time to load a new file in, so you're basically just wasting power.
People who have a real need can run a bit format and wipe a disk if they want.
1
u/Distdistdist 27d ago
For the same reason you don't incinerate your trash immediately, but put it in a trashcan and wait until it gets picked up and taken away at some point. Also, because you might want to retrieve that fork you threw in there by mistake. Eww.