Most Digital Content Not Stable

brunes69 writes "The CBC is running an article profiling the problems with archiving digital data in New Brunswick's provincial archives. Quote from the story: 'I've had audio tape come into the archives, for example, that had been submerged in water in floods and the tape was so swollen it went off the reel, and yet we were able to recover that. We were able to take that off and dry it out and play it back. If a CD had one-tenth of one per cent of the damage on one of those reels, it wouldn't play, period. The whole thing would be corrupted'. Given the difficulties with preserving digital data, is it really the medium we should be using for archival purposes?"

Re:Multiple identical copies?

[t00le]

Any good backup strategy will have multiple media types, so CD/DVD should not be your primary backup media type. If you prefer to have an medium for fast access, then it is still viable. As long as it is not your primary media type, which should be something with tried-and-true longevity.

wring recovery method

[Red Flayer]

If a CD had one-tenth of one per cent of the damage on one of those reels, it wouldn't play, period.

That's because you're trying to optically read through the damaged part. It is possible to recover data from damaged discs, as long as only the coating (and not the reflective surface) is damaged. It is quite possible to polish the surface and read the data, or even to fill in some of the damage and repolish for reading.

Just because it's harder to recover the data doesn't mean it's impossible.

Of course, anyone using CDs or DVDs for large data backup must have a lot of interns to do the disc swapping.

have people already forgotten?

[phantomfive]

Have people already forgotten the advantage of digital? If you have an analog tape, every time you make a copy of it, the quality will be degraded. But with digital, you can make a million copies and the final copy will be the byte by byte equivilent of the original. So what if CDs only last 10 years before becoming unusable? You can make another copy! So what if this guy wouldn't have been able to recover after physical damage to his media....if it was important, he should have had digital offsite backups! And those backups would have been 100% equivelent to the originals.

We can take this seriously.

[Lethyos]

Emphasizing the “I” in RAID [8k.com].

Re:Like what?

[t00le]

Hrmm,

DLT
reel-to-reel
Mini8mm
SAN
CD/DVD
etc...

Depends on how deep your pockets go and your calculation for the value of the data if lost. You are doing the math on loss of data, riggghhhhttt?

Re:wring recovery method

[Criffer]

Exactly. If you try to put a bent CD into a CD drive, you're obviously not going to be able to read it. But that doesn't mean its not recoverable.

To recover data from a CD, you can simply photograph it at high enough resolution. Even with huge scratches, even with parts of the disc physically missing, you can recover the data exactly as it was encoded. How? Reed Solomon code [wikipedia.org] .
Quoth wikipedia:

The result is a CIRC that can completely correct error bursts up to 4000 bits, or about 2.5 mm on the disc surface. This code is so strong that most CD playback errors are almost certainly caused by tracking errors that cause the laser to jump track, not by uncorrectable error bursts

Apples and Oranges

[Overzeetop]

The tape had analog data on it. Analog, as we all know from years of television and radio, is very forgiving of damage. CDs are digital data. There is error correction, but for normal playback/reading devices there is a limit beyond which they simply give up trying. The data is perfect or its gone for those machines.

Sad to say, tape dies too.

What is more interesting is the use of compression (and rights management, though if your originals are encrypted you deserve to get screwed - physical security comes first). With analog and simple stream encoding of time domain data (such as audio recordings) much data can be recovered using an external benchmark for the time code. Compress that data and lose your parity and you're totally hosed.

I've never been a proponent of compressed or encoded backups. Sure they save space and add a layer of "security", but that comes at the cost of flexibility should damage occur.

Of course, as has certainly already been mentioned - with digital data, you have the luxury of making multiple perfect copies as well as the ability to perform automated checks of that data, mostly possible without user interaction necessary.

Othwise, stone tablets have the best track record so far, though the storage density is a bit on the light (or should I say heavy?) side.

Re:That's nothing, think of DRM

[saforrest]

While whites did enough evil, like stealing the whole country, American Indian writing systems were actually developed by missionaries.

I think that was the point behind "depending on how you define American" -- the GP was referring to the urbanized cultures of Mexico, Central and South America that had writing systems that they were forced to give up along with the rest of their culture.

Re:They could try harder

[clickclickdrone]

a special "archaeological" CD player

I believe they exist already - just as there exist devices for reading fragments of shattered hard drives. Forensic data recovery experts have some pretty funky kit at their disposal.

Sorry to spoil the fun (VXA tape format)

[mihalis]

I know I'm offtopic, injecting facts into this debate, but I thought it might be interesting to bring up the VXA tape format. It allegedly survives all kinds of abuse like freezing, see Freezing Test [exabyte.com]

I have never tried these drives, and would love to hear from someone independent who has.

Re:1% = Total Loss?

[the eric conspiracy]

CD Paranoia. I've used it to recover CDs that a $1000 player choked on.

Re:It's the messanger, not the message

[mypalmike]

Nothing is wrong with digital it all depends on the medium.

It depends on the content as well. Content that is inherently analog tends to be more 'robust' in analog form. For instance, in the military they say [wired.com], "A computer with a bullet in it is just a paperweight. A map with a bullet in it is still a map."

Re:That's nothing, think of DRM

[UncleTogie]

To them, it was impossible to 'claim' their land -- since they didn't consider it 'their' land.

Best summed up by Chief Seattle, in 1854: "This we know: the earth does not belong to man, man belongs to the earth. All things are connected like the blood that unites us all. Man did not weave the web of life, he is merely a strand in it. Whatever he does to the web, he does to himself."

There was life before CDs.

[Kadin2048]

That's the first real main stream didgital format and it has not yet reached the point where we have to move data off of it for fear of it being unreadable.

Erm ... say again?

There are terabytes (quite literally tons) of data sitting around on everything from old 7- and 9-track 1/2" open reel tape, to old 8" and 5-1/4" floppies, and other formats that are basically dead. [I'm not familiar with anything older than that, but I'm sure there are some real greybeards around that could enlighten you as to what came after punchcards but before the vac-column tape drives.] The only saving grace of those formats is that if you can find a reader, there's a chance it might either still work, or could be made to work, if you could find a compatible computer to interface it to (because the machines themselves were built pretty well; they were still viewed as industrial equipment of a sort, rather than consumer electronics). But the expense of doing that would be enormous -- the people who know how to maintain, and increasingly to operate, those things are retiring and becoming hard to find.

And analog formats aren't exactly immune, either. Where I used to work, we had several boxes of old video recordings on 2" quad [wikipedia.org] that we were storing for preservation purposes, but couldn't afford to have transferred to another medium (despite the obvious: that the longer you wait, the more expensive it's going to get if you ever do really want it). That format was used for over 20 years; there's got to be thousands of hours of it sitting around.

Even if you define 'mainstream' to be something that an average person could afford, CDs certainly weren't first; lots of people had PCs with various types of digital storage.

But to only focus on 'mainstream' formats misses the point entirely. Stuff that's been distributed out to millions of people isn't what's at risk of disappearing; it's the original source material (think NASA's Apollo videos), or information that's naturally stored in big 'silos' (think public records) that's really at risk, and those have been stored in a plethora of formats, digital and analog, over the past 50-75 years, which are difficult to work with today.