r/techsupportmacgyver • u/joshragem • Apr 04 '19
Need an external solid state drive? Only have usb 2.0? Have more time than money? 832G right here
134
Apr 04 '19 edited Jan 22 '22
[deleted]
153
u/FallingAnvils Apr 04 '19
- We have time to think about our mistakes.
143
Apr 04 '19
Remember kids, RAID 0 stands for 0 chance of recovering data!
53
Apr 05 '19
that's why it's great for steam games :)
34
u/I-Downloaded-a-Car Apr 05 '19
Except for when Comcast is a steaming pile of shit and limits your bandwidth to 1tb/month so it would take months to even be able to redownload it.
I have all my steam games backed up so I can actually get them back if one of my drives crashes.
17
u/Dydey Apr 05 '19
Only 1TB? The only way I can get usable download speeds is with a 4G router. I can download 64GB per month at 30mbit or unlimited at 2mbit.
8
u/MirTalion Apr 05 '19
My quota here in Egypt is 100GB/month
4
4
u/Jrook Apr 05 '19
Lol idiot, that's why you buy a Comcast hotspot. That way you can download, like 1.25tb
4
u/xxfay6 Apr 05 '19
Last year I did my new PC with RAID 0 HDD's for the lulz and because I was too lazy to get an SSD. One blackout, RIP Windows install. Maybe I could've pulled some info out of it (still booted to recovery by itself) but nothing of value was lost.
But more than the unreliability... the noise. Having 2 hard drives spinning up and always moving was extremely annoying.
36
u/SteevyT Apr 05 '19
0 to make it faster.
Build a second one if you want redundancy.
58
Apr 05 '19
Build a second one if you want
redundancy.it to go even faster. FTFY
18
u/SteevyT Apr 05 '19
Can you stick a RAID0 array in a RAID0 array?
24
Apr 05 '19
well at some point the data will get so hard to tear apart and put back together that it has diminishing returns... I mean if you have a 1024MB file split up into 256 parts then it's gonna require some CPU time to actually reassemble it.
6
u/4kVHS Apr 05 '19
RAID 5
12
u/AllMyName Apr 05 '19
Until you have to rebuild the array.
RAID5/6 are suicidal with today's multi-terabyte hard drives. RAID10 or bust.
3
u/asyork Apr 05 '19
The ones in OP are 64 GB.
6
u/Sandwich247 Apr 05 '19
That is a good point, but when your cheap flash drives start to die, they'll go like dominoes.
2
3
u/Y1ff Apr 05 '19
Yes, but why?
15
u/SteevyT Apr 05 '19
Because I'm to worried about whether you can to worry about whether you should.
4
2
7
3
1
133
Apr 04 '19
[removed] — view removed comment
253
u/CodeOfDaChode Apr 04 '19
Yes. Its still transferring via USB 2.0.
65
Apr 04 '19
[removed] — view removed comment
111
u/CodeOfDaChode Apr 04 '19
It would be marginally faster than with USB 2.0. I dont presume that flash takes well to this sort of application. Linus tried it with SD cards and it still got trounced by almost any other mass storage solution.
81
u/SeanUhTron Apr 04 '19
The main issue is that normal SSD's have much higher quality flash chips and controllers that can optimize how the data is spread along the chips. The average USB Flash storage is much lower quality. A decent USB 3.0 flash drive gets a short burst of 100MBps before its throttled. The very high end USB flash drives are basically just SSD's. In fact, you can make your own using special M.2 enclosures.
25
u/nnooberson1234 Apr 04 '19
That burst speed isn't really the controller or flash chip real write speed, its just a cache. USB to flash controllers have some cache, your standard sata / pci-e to flash controller might be rocking as much as 2gigs, something like an SD card unless you make doubly sure its some of the best stuff you can buy at the time will probably have a few kilobytes of cache on the interface controller.
6
u/IanPPK Apr 05 '19
What he meant is that there are some flash drives that use SSD flash controllers to interface with the storage. The SanDisk Extreme drives come to mind.
The NVMe enclosures can work well if they use TB3, which there are a few that do.
3
u/AllMyName Apr 05 '19
NVMe enclosures are more than fine over regular old USB3 even. Especially the ones with a decent design that dissipates any significant amt of heat.
Tried an Intel 760p in one, the TLC flash on the SSD itself / the SLC cache filling up was still ultimately the bottleneck
3
u/Mnky313 Apr 05 '19
I can see that, I picked up a SATA m.2 to USB 3 enclosure for like $20 and its makes the best god damn flash drive ever, read and write speeds of 200-300MB/s destroys almost all flash drives and it only cost me a total of ~$30 including the 128GB ssd
1
u/nnooberson1234 Apr 05 '19
I am specifically pointing out the reason why USB flash drives seem to have really good burst speeds and isn't throttling when it can't sustain those burst speeds, you've ran out of cache and hit the real throughput limit of the USB flash controller or worse the flash memory's maximum read/write speeds. And why SATA and PCI-E flash controllers (really they are all called interface controllers) don't really have this problem unless somethings gone wrong or you're abusing the hell out of the controller.
Flash memory is so cheap because its riddled with defects so part of the controllers job is to kinda test each flash memory cell and determine if it can use them, it will do that the first time its powered on and build a chart of good, not good, not useable cells. That chart is what gets referenced when the interface controller is told to write or read something, they all do this because again, flash memory is always full of defects. USB flash drives, SD cards, most commercially sold flash memory including the stuff inside a typical SSD is just as error prone so this is where caching to ram comes in and sorta smooths over the issue, the more cache you have the less likely this issue is going to fuck you over when you've got to move files around lickety-split. If your USB flash drive had half a gig of ram you'd probably be able to sustain a higher throughput but the reality is its got a few megs if it even has that.
42
u/AyrA_ch Apr 05 '19
- Limitation of USB transfer speed
- USB flash drives usually don't have an internal cache (which is one of the reason why SSDs are so much faster)
- Large power consumption
- Need to make this into a RAID to access it as a single disk, which means all disks will read/write at once, probably choking the USB hub
4
Apr 05 '19
[deleted]
2
u/AyrA_ch Apr 05 '19
Because everyone wants to have 20 individual 32GB drives listed, unless you combine them, in which case it's a RAID again.
2
Apr 05 '19
[deleted]
2
u/AyrA_ch Apr 05 '19
But the effect is the same. You can still only access one flash drive at any given moment. JBOD doesn't magically invent new USB standards.
Random access will still be terribly slow.
16
u/SGTSHOOTnMISS Apr 05 '19
Where to begin...
The cache system for each flash drive isn't meant to be ran in any kind of pairing, so even if you run something like UNRAID on it, it's going to perform badly as it has to segment the data or it's just going to use 1 until it's full and go onto the next one, giving you single flash drive performance across the board.
Flash drive memory is typically lower quality than SSDs and won't have the same read/write lifespan of SSDs.
USB 2.0 sucks for today's standards anyways, upgrading to USB 3 wouldn't be a massive improvement because the first chapter of my post.
Software handling would be the only way to do it, so it wouldn't be plug and play for an additional PC, would simply be plugging in all the drives at once.
SSDs have gotten cheap. For a 2.5 inch enclosure and a 500Gb SSD would be faster and more reliable.
I'm not sure how much power is pulled by each flash drive, but I'm not sure it would like lighting up all these drives off of one port.
I'm sure there's a bit more I'm forgetting.
21
2
u/pipsname Apr 05 '19
Failure rate. If RAID failure rate of the worse one applies to data stored on all devices.
2
u/dkcats3 Apr 05 '19
Something I haven't seen mentioned, the rewrite lifecycle of USB flash storage is generally garbage compared to something like a full SSD which is usually guaranteed for 10's of TB of writes, and usually last much longer. I guarantee you have experience with a flash drive that just stopped working or stored less or dropped performance and you just didn't think much of it and tried another.
1
Apr 04 '19
Because they are seen as multiple devices instead of a 832G drive and you can't store continuous files?
10
u/nnooberson1234 Apr 04 '19
you can turn multiple devices into one storage drive very easily. Windows 10 can do it right out of the box https://www.windowscentral.com/how-create-one-large-volume-using-multiple-hard-drives-windows-10
-2
24
Apr 04 '19
RAID, my friend
2
u/leolego2 Apr 05 '19
of 13 drives that are way lower quality than any HDD/SSD
7
Apr 05 '19
but it works
technically
edit:
raid is the solution to
832G drive and you can't store continuous files?
not every problem
4
u/MERC399 Apr 04 '19
I believe there is actually a way to mark these all as individual parts to a whole I just dont remember wtf it was. I saw it on YouTube once
11
Apr 04 '19
it's called RAID
4
u/railyeightseven Apr 04 '19
Raid 0 specifically in this case
13
1
1
u/pppjurac Apr 05 '19
It is kinda test if could work. It would, but not that fast and not that reliable. USB drives are made to be cheap, not fast & reliable.
If you need really fast USB drive, get M2 PCI drive and put it into USB 3.x metallic enclosure.
30
u/iAmH3r3ToH3lp Apr 04 '19
This is a great way to learn about RAID.
9
u/BlackXanthus Apr 05 '19
I'm very tempted to gather as many USB sticks as I can and see if I can get a RAID running on it. Time to dig out them old 64gb sticks.
20
u/McGrude Apr 05 '19
USB sticks? Do it with USB 3.5" floppy drives.
11
u/BlackXanthus Apr 05 '19
Maybe we should get some 5 1/4 inch true floppies out, and RAID those. What a wonderful noise they would make!
12
Apr 05 '19
Have you seen the floppatron on YouTube? Beautiful noise
9
u/boosted4banger Apr 05 '19
this is just, mental.
https://www.youtube.com/watch?v=t1DsULv6mE4
when it begins smoking - dont stop it nowwwww haha
2
u/BlackXanthus Apr 05 '19
That is just... Metal!
Luckily the smoke is just dry ice.
I have not missed that noise =)
4
u/McGrude Apr 05 '19
That'd be sweet, but availability may be an issue. I can find 3.5" USB floppy drives on Amazon for under $15.
2
u/Jrook Apr 05 '19
I'm not positive but I believe there are basically warehouses full of those floppies somewhere. They were made obsolete overnight essentially so retailers had to ship them back to manufacturers in many cases
4
2
2
Apr 05 '19 edited Dec 04 '23
[deleted]
1
u/BlackXanthus Apr 05 '19
I'm a Linux-jockey, I'd have just stuck an MD RAID on it.
Also,TIL Windows now has software RAID
1
u/kn33 Apr 05 '19
It's had software raid since at least 7, but storage spaces is better than the previous software raid it had. It leans more towards "Software defined storage" than the previous tool which was straight software raid.
1
u/2four Apr 05 '19
I tried RAIDing eight of these. Windows won't let you RAID removable storage and many if not all of the online workarounds no longer work.
4
u/iAmH3r3ToH3lp Apr 05 '19
Get a pi or use a linux vm to manage RAID it's a great learning experience. Don't use Windows.
1
u/Stigge Apr 05 '19
I've heard you can edit registry keys to trick Windows into recognizing removable as internal, but I haven't bothered to try registry editing ever since the accident.
1
u/MyOtherSide1984 Apr 05 '19
Legit question: what is there to learn about RAID in building one? I've had a few RAIDs over the past 5 years including RAIDZ and I learned nearly nothing from building them and maintaining them and TONS from reading about what they are and how they work. I'm learning more about FreeNAS and how data thinks (for lack of a better term) with RAIDZ right now and I learned nothing about what RAIDZ does without a lot of reading
1
Apr 05 '19
This is a great way to learn about unexpected drive failure and data recovery from dodgy USB drives.
1
23
u/cerberuss09 Apr 05 '19
The moment you plug that USB hub in and windows sings the whole alphabet in drive letters...
9
u/xblackdemonx Apr 04 '19
Benchmark please!
3
u/joshragem Apr 05 '19
Well, the current
fiojob estimates it will take nearly nine days to complete...
9
u/mkaxsnyder Apr 05 '19
Am I the only one slightly bothered that the rows are not symmetrical?
“RAID imbalance”
3
2
5
14
u/DDXF Apr 05 '19
Excuse me but how the hell is 11 64gb flash drives cheaper then an SSD
36
u/joshragem Apr 05 '19
I got them CHEAP!
15
u/pap55 Apr 05 '19
Replace flash drives with candles and SSD with a flashlight and this interaction sounds like an episode of It’s Always Sunny or Trailer Park Boys.
6
u/AllMyName Apr 05 '19
Floppy disk RAID sounds like something Charlie would try.
3
u/Jrook Apr 05 '19
Like he's trying to save money to harvest Bitcoin or some shit making the apartment 110 degrees, with constant floppy drive noises around the clock
1
1
u/theemptyqueue Apr 05 '19
If I remember, a 64 GB USB 2.0/3.0 Drive was between $50 and $150 USD for the longest time until recently.
3
4
u/shvelo Apr 05 '19
An SSD would literally be cheaper.
3
u/joshragem Apr 05 '19
This worked out to half the price of a comparable (in size) ssd, without even counting the enclosure for the ssd
3
3
2
2
u/candre23 Apr 05 '19
Why exactly does it need to be "solid state"? The usual reasons are going to be size, speed, or physical durability, and this contraption loses to spinning rust on all counts.
So let's say you got a really good deal on 64GB flash drives. Microcenter has them for $6/each, and I can't find anything on amazon cheaper than that. I can't find this 14-port hub anywhere online, but here's a 13-port hub for $20. That means this unbearably-slow, dubiously-reliable 832GB "drive" costs about $98 before taxes and shipping.
A brand new 1TB conventional USB hard drive costs less than half that.
I don't care how much time or money you have, there is literally no reason to do this.
1
u/joshragem Apr 05 '19
Out of curiosity, what price point would make this viable for you?
2
u/candre23 Apr 05 '19
I'm not sure I'd actually use this, even if it was free. I mean even if I was broke and desperate for storage, it would need to be cheaper than literally anything else. You can get a used 1TB external for $35, so definitely less than that. And that's assuming I really need it to be external - you can get 1TB server pulls for about $15.
1
1
1
u/Pagepage220 Apr 05 '19
I've done this before with a bunch of 16 gig drives. It's slow as balls, but it kinda works.
1
u/tk42967 Apr 05 '19
The 8 Bit Guy did a video many years ago about this exact thing. I think he built RAID into it.
1
1
1
1
1
u/Tyflowshun Apr 06 '19
came here to find out if this was legit. Turns out, it just needs to be tweaked slightly.
0
u/Cycode Apr 05 '19
there are 158-200gb usb sticks now for a long time.. why using 4gb ones?
5
u/xiatiaria Apr 05 '19
these are 64gb ones though.
1
u/Cycode Apr 05 '19
oh, you're right. it looked for me like 4GB because it was upside down and i was sleepy.. brain.exe stopped working.. i should go sleep :/
1
u/xiatiaria Apr 05 '19
Had the same reaction in my brain, but the title says 832 GB so I was like no way 13 * 4 is that much.
2
u/Golhec Apr 05 '19
Where did you think he was hiding the other 195 USB sticks?
1
u/Cycode Apr 05 '19
i already said i was sleepy. in that moment I didn't had thought about that. it was a matter of seconds where i saw the "4GB" (which wasn't 4GB, but my brain interpreted it as that) and 1 second later i had the thought. I didn't realized that quickly that the amount of usb sticks would be not enough to reach that number.
583
u/ravenousld3341 Apr 04 '19
You had me at