Unless the 1.6petabytes is all photos you have ever taken of people and all the photos and videos everyone has ever taken of you and all your family, I hope there’s a way for a person to wrap their mind around having 1.6petabits. maybe it’s a big text file that draws your name from random text characters to the order of 1.6petabits. it could be mostly just zeros.
Since the realistic competitor here is probably magnetic tape, current-generation (LTO9) media can transfer at around 400MB/s, taking 12 hours and change to fill an 18TB tape.
Earlier archival optical disk formats (https://news.panasonic.com/global/stories/798) claimed 360MB/s, but I believe that is six, double-sided discs writing both sides simultaneously, so 30MB/s per stream. Filling the same six (300GB) discs would take about an hour and a half.
Building the library to handle and read/write in bulk is always the issue though. The above optical system fit 1.9PB in the space of a server rack (and I didn't see any options to expand further when that was current technology), and by the looks is 7 units that each can be writing a set of discs (call that 2.5GB/s total).
In the same single rack you'd fit 560 LTO tapes (10.1PB for LTO9) and 21 drives (8.4GB/s).
So they have a bit of catching up to do, especially with LTO10 (due in the next year or so) doubling the capacity and further increasing the throughput.
There's also the small matter that every one of these massive increases in optical disc capacity in recent years has turned out to be vapourware. I mean I don't doubt that they will achieve it someday, but they always seem to go nowhere.
that’s neat and all but the headline reads a lot like those usb batteries advertisements claiming 15000 mAh capacity - the big number means it’s better! but lol no it’s only 15amps.
200tb enterprise level storage is good - if it has a really long shelf life, like 10+ years. your critical backups are only good if the data doesnt degrade or the storage media doesnt fail when you’re doing a DR.
I’d said over on the Old Place back during the Blu-Ray/HD-DVD wsrs that people really liked their 7" optical media. I got down voted to hell for it then, but I’m glad to see I wasn’t totally wrong.
Cheap, high density media has its applications. Tape is still the preferred long-term storage medium for backups in a lot of industy sectors because still stores gobs of data, it’s dirt cheap, compact, light and it transports easily. If you don’t need it to be fast, or you’re regularly producing large scale data sets that are essentially disposable after some time, then it’s a good compromise.
No reason this tech couldn’t step into that niche when it hits the right price point.
Yea but the tape is likely to last the 20-30 year estimate. You couldn’t say the same about HDDs especially the helium sealed ones.
Whether the tape drive will survive as well is another question but between the simpler mechanism, a drive 2 generations ahead can still read the tape, parts inter-compatibility if you needed to frankenstein an older drive with new rollers and motors and just plain buying and keeping drives sealed in storage as new-old-stock ahead of time. You have a few options to choose from.
Where as with HDDs you may have to repair each one. The helium ones you may have to re-gas.
Tape sounds like a better long term archival/backup approach.
Long term yes, i absolutely agree with you, however, you sure you’ll survive next 20-30 years? For datahoarders and home server solutions 20tb hdd is better, while as you’ve said and i agree with you that for long term storage tape is better, but with speed of web development nowadays, archived data will become obsolete blazingly fast, so lto tapes is very special case while HDDs offer more memory for the same price and designed for constant rewrites and have more layman interface (sata) so they better for your average home server data hoarder
Well I’m in my mid 20’s so I’m hoping for at least that long :). No I won’t likely need alot of what I store to last that long although I am a member of r/DataHoarder (not sure if they’re on lemmy yet) but for a few items like family photos/videos it’s nice to have it written in a way that I can mostly just set and forget. With the standardization and open source implementation of LTFS you have even less worry about having the software to read it in the future. A SAS IT mode HBA and linux with a git clone of the LTFS repo is all you need.
In terms of cost the drive was very expensive ($2500 NOS from eBay US) but if you treat that as the one off entry cost, the tapes are cheaper for me to buy than the equivalent in HDDs here in Australia. That’s comparing ~$460 20tb EXOS HDDs from serverpartdeals.com to $43 x 8 = $344 2.5TB LTO-6 from stutchdata.com.au.
Also I store the tapes in IP67 boxes from bunnings along with a pack of desiccant and put the boxes in a cool but damp area. Don’t really have alot of choice where I live. It’s either that or hot daily temperature swings. Basement vs attic/garage.
I hope that’s enough to store them correctly environmentally speaking. I am in the process of working out how to clean family VHS tapes that were not stored correctly and that’s not an operation I want to revisit. An extended project is to make 900mhz button cell humidity/temperature monitors to notify me when desiccant has expired.
This may seem excessive but I would argue most don’t do enough in an age where more and more is being stored digitally as the only copy rather than print, etc. I feel this is a small price to pay to keep the still more compact and convenient all digital lifestyle without the data loss issues most people experience. The drive was expensive to buy into but with how little I use it I hope it’s going to last a long time.
Yeah i agree with you, most people don’t do archives of data, and if you do archive then lto tapes is the best, hovewer most data hoarders just like me just wanna hoard data, just as simple as that, no deep meaning, and for casual hoarding server stocked with hdds with zfs and deduplication and compression is just the right fit, however, i feel you, data needs to be archived for future since corporations isn’t interested in this, and games, videos, music, and many many more other things i can’t even imagine just disappearing without archiving, so those who archives is MVP, but most data hoarders is just guys and gals who just want to have autonomous library of media on their home server/lab, so while i feel and understand and support your idea, what I’ve been trying to say is, most people want autonomous library and as cheap as possible, HDDs are better for it while for archive purposes lto is the best as we agreed in our previous comments, and about surviving, I’ve seen enough cases when people die just like that and current geo political situation ain’t helping, also if you frequently read bad news publics then you’ll know that literary anything may happen, car crash, aneurysm, some rando going on a rampage, some gas explosion, some fire, some drunk courageous blokes wanting to have violent fun while you wandering in evening, so i prefer just setting long term goals and doing them, and yes, I’m mid 20s just like you, however seeing how people can just die in their daily life, i ain’t betting on anything, just doing my goals, that’s it
I suppose with that much data capacity they could halve the storage and add redundancy. My question is will it only have 1 reading head? That much data is going to take a very long time to read, unless they’re doing multiple layers at a time,
I’m not familiar with it either, but I’d say that using RAID on a single disc is silly… There’s a good reason it’s not a common practice on single HDDs.
A scratch on the disc usually means many scratches on the disc.
What of you put it on an enclosure that has the disk(potentially even more stacked on top of each other) plus all the hardware needed to read the disk. Then all you would need is to provide power and plug in a data cable.
Or like a UMD or mini disc? Still have to insert it into something to read and write, but the discs themselves are enclosed and protected unlike CDs and DVDs and Blu-ray. Basically they’re floppy disks, but instead of magnetic tape inside the shell, it’s an optical disc.
I like this and for what it will likely cost I’d hope would have it. Other than scratches, dust, oils from touching, light and other contaminates are the biggest threat to longevity.
Believe it or not, first gen DVD-RAM came exactly like this. But manufacturers cheaped out / wanted the drives to be more easily compatible with CDs. So the caddys were scrapped.
Due to the caddy nature I believe there were plans or limited availability of double-sided disks. That would have made it so much more appealing I think.
Here I thought that was because nearly no one uses them anymore. The large volume of folks who didn’t coddle their DVDs are Netflix subscribers now, the few people who do still bother to buy movies or games on disc are the folks who care about them, and thus don’t leave them on the TV stand to get scratched.
Listen, the idea isn’t that you’ll have a walkman that has every YouTube video you’ll ever watch on it.
It’s that you’d backup an entire fucking enterprise on one disc. Schedule it daily. Pay the support team to swap the disc out every night. Who needs infrastructure for ransomware, we got DISCS!
This is great, I have a NAS and I still get slow data corruption for my long term data even using bit-rot resilient file systems. I would like a way to back up 5Tb of photos and videos on a single disc to bury somewhere for long term (decades) storage.
So optical drives will make a come back perhaps? Guess this means we’ll have malware similar to in the early days where it would spontaneously open your CD-ROM drive “as a cupholder.”
Anytime you get to that length, you always have to think about whether or not someone will have a drive to read it, a computer that it works on, and matching programs to decode the data. Think about some of the formats we had in the 70’s and 80’s and how often people actually have that hardware and software in working order now.
That’s the only hurdle if you have the software and decoding both of which are emulateable. Which wouldn’t be overly hard to reverse engineer a connector if you have everything else…
Assuming the software isn’t lost, then yeah, typically it can be emulated or reverse engineered to work.
The bigger hurdle is the hardware, especially if the encoding of the data was proprietary, meaning that even if you could get a reading without it, you’d still need to figure out how to decode it into useful data
Think about some of the formats we had in the 70’s and 80’s and how often people actually have that hardware and software in working order now.
Well yea, but it’s a matter of funding and business/government desire. 99% of the time the only people who care about accessing things that old are hobbyists and enthusiasts.
If something critical to a fortune 500 company or government was stored on it and they needed it they would have the means to contract out a specialty one off device just to read it (Or contract out to a very pricey data recovery shop)
And software is software, we can still run 70s and 80s software through a myriad of virtualization technologies fairly easily and cheaply.
You want to make some money? Start manufacturing microfiche readers. There was a brief time in the 20th century where microfilm and microfiche was all the rage for archiving and even publishing technical documents, and now there’s a lot of data people need for various reasons and no device to retrieve it on because they all got put in a room in the back of a library and got kicked in when someone backed into the room carrying a heavy box.
Real, good quality, factory-made discs, maybe. Anything else (from bad quality factory stuff to writable discs), not so much. And backups where not done on factory-pressed discs.
Blu-Rays were more of a pain because of the format itself; Handbrake itself wouldn’t do the job, I had to use MakeMKV to get a huge mkv file then wash it through Handbrake to compress it to an mp4. Not a single one failed.
Movies on DVD, out of ~300 discs, I had a total of 6 fail because the discs are somehow damaged, most were visibly scratched and wouldn’t play back in a normal DVD player either.
TV shows on DVD, out of ~150 discs, ~40 of them partially or totally failed, many had visible disc rot. And there was definitely a pattern that boils down to “cheaper discs tended to fail.” Older discs from earlier in the format’s life proved more reliable, I think because, for example, my copy of Friends was purchased in the mid-2000s relatively early in the “TV shows on DVD for binge watching” era, some 60 discs in total, no failures. Smaller runs of shows that not a lot of people bought that were kind of plunked out on DVD for the nine people that bought them like Kolchak: The Night Stalker or The Greatest American Hero? 50% failure rate. An interesting one is my copy of Stargate SG-1. I own some seasons from an earlier pressing that came in individual standard plastic cases in a cardboard box, you know what I mean? Those were reliable, only one disc failed because of scratches caused by mishandling. I own some seasons from a later re-release in those slimmer 5-discs-in-a-cardboard-foldy-thing, and more than half of those are unplayable due to disc rot.
Meanwhile I have CDs made in the 80’s that still play just fine.
Optical discs are already incredibly resistant and shouldn’t be expected to fail in your lifetime. Most of the times they do, it’s either old media (cd and dvd both had physical flaws in design), damage, or mistakes in manufacturing.
There’s really no reason for the discs to degrade. It’s just stamped plastic.
Pressed optical discs will last a very long time. The lifetime of burned discs depends on the type of dye that’s used to store the data. Many of the early CD-R’s would get corrupted after a few years, but that was solved a long time ago.
When they say plastic takes [huge number] of years to decompose, they’re talking about how long it takes to disappear completely. The usable lifetime for most plastic objects seems to be only a few decades. (I don’t know about the specific plastic they use for optical discs, though.)
As an optical media enthusiast, I’ve done a fair amount of research into how, why, and when discs fail. Because the discs use two or more polycarbonate layers pressed together, moisture can sometimes work its way between the layers and speed up degradation, especially if a disc has been overly flexed at the center. Heat and UV can also speed up degradation.
Another problem is that plastic is petroleum-based and it breaks down over time. A lot of people think that the reflective layer (the metal layer) is actually the data layer but it almost never is. The data layer itself is polycarbonate, sandwiched between the reflective layers and more polycarbonate layers.
The newer discs like blu-ray movies are made with better plastics that should last at least 100 years. Depending on the dye layer of writable and rewritable blu-rays, they should last either at least 25 years or 100 years.
Add comment