Please create an account to participate in the Slashdot moderation system

 


Forgot your password?
Close
typodupeerror
×
Technology Software Hardware Linux

MiniDV As A Backup Medium 200

Posted by Hemos from the must-do-it dept.
MiniDVfanatic writes "Last year I read an article in Slashdot about using MiniDV as a backup medium. Now I've found a recent article about the same topic in bulmalug.net. It's in Spanish, and it just talks about dvbackup, the same tool recommended in some old Slashdot posts, but it adds some interesting ideas such as data compression and interleaved Reed Solomon for error detection and correction, using a tool called rsbep. According to the author's tests, this should let you backup 13GB in a cheap MiniDV tape in a safe manner, and restore it all later. Example commands provided, with simple workarounds for slow machines which cannot feed the tape at 3.6MB per second." You can fish the translation, for those who need - I was able to get by with my high school Spanish. Pretty cool stuff.
This discussion has been archived. No new comments can be posted.

MiniDV As A Backup Medium More

MiniDV As A Backup Medium

Comments Filter:

  • What about speed? (Score:5, Insightful)

    by Max Romantschuk ( 132276 ) <max@romantschuk.fi> on Monday March 03, 2003 @09:07AM (#5423313) Homepage
    I've used tape backups... Sure they work, but when you want a certain file from the middle of a tape it sure is no picnic.

    Harddisks are really cheap these days, CD-Rs are also cheap and writable DVDs are becoming an option... Does it pay off?

    • Re:What about speed? (Score:5, Informative)

      by eXtro ( 258933 ) on Monday March 03, 2003 @09:18AM (#5423367) Homepage
      What's the longevity of a CD-R, especially the really cheap ones? What's the longevity of a DVD-R, especially the cheap ones? A tape is more of an archival medium while CD-R and the like are more of a medium term storage format. I have data from my thesis stored from about 10 years ago and have recently (last week as it turns out) extracted data from it. I've had CD-R's that have failed after only a couple of years.
      • I have data from my thesis stored from about 10 years ago and have recently (last week as it turns out) extracted data from it. I've had CD-R's that have failed after only a couple of years.

        A very good point. I would however dear to claim that there are probably cheap DV-tapes which could easily suffer the same fate as cheap CD-Rs.

        Then again, I might be wrong :)
        • My brother (The Evil Genius) always told me the only safe, sane way to store data over the long term is on punch cards.

          Think about it.
        • The expected shelf life of a DV or MiniDV tape is at least a decade. Consumers don't buy most of the DV tapes, they keep reusing one or two and outputting the results to VHS. Thus, they need tapes that will last lots of use. Professionals buy most of the tapes (use once, store away) and expect to use the tape as a high quality master for years (decades?) to come. Because of this, and the trade practice of 'outing' inferior tape makers, you don't have to worry as much about your DV tape quality.
        • There are definently weak DV-tapes. Any dv/dvcam operator will tell your which brands work and doesn't work with hers/his particular setup...
          Some tapes will cause digital-dropouts and worse almost instantly when used in certain cameras while work perfectly in others...

          There seem to be a lack of quality in some tapes or there's some misunderstandings between some camera-makers and some tape-producers ;-).

        • No! miniDV is NOT a good long term storage format. miniDV is the consumer version of the DV format. Professionals use the thicker, higher quality tape formats of DVCam and DVCPro for long-term archival needs.

          Also, no DV formats are designed to be 100% bit safe. A lightweight video codec like DV can handle an off bit here and there without ruining the project. Just a little visual glitch here and there on a single frame, and pro decks do a good job of interpolating the missing data so the problem isn't too obvious. Not really possible with a compressed data bitstream.

        • Even expensive DV tapes used to be pretty unreliable. I'm not sure if they still are. I used Panasonic mini-DV tapes (at the time, $15 apiece) to shoot a movie. At least two of the tapes had missing frames upon replay a few minutes later. That's two out of... oh, ten. Not at the ends of the tape, either. Randomly distributed. Thankfully, I caught the most critical one on the day of the shoot and was able to go back and retake the scene. (The others were covered by using different camera angles for those lines.)

          I have two Panasonic DV tapes recorded as recently as December of 2000 with dozens of missing frames caused by severe dropouts. It was one of only two on-tape copies of the master for said movie, and the only copy that's here in California.

          Thankfully, when I made the copy of the original, I kept the files on a hard drive. The hard drive data, unlike the DV backup, is intact; the DV original, shot to hell.

          Hopefully tapes by other vendors (TDK, Fuji, etc.) are more reliable, but even still, the thought of trusting any valuable data to be adequately backed up by mini-DV tapes makes me shudder.
      • You bring up interesting points, but if one is chosing any backup solution, I doubt one would find the cheapest media... what happens when you chose a cheap tape rather than the nice one that you chose 10 years ago? As far as I have seen, magnetic media dies a lot faster than optical media, and it is simply used because of the massive storage possibility.

        Tape strikes me as a redundancy plan for short term backup on a daily to weekly scale, however if you are going to expect something to last 'forever' other forms of storage may be better. You say that you've had CD-Rs fail, but have any failed that had the same ammount of use as the tape that you just dug out of the closet?

        People mention DVD-Rs which as far as I have seen work fine as long as you don't handle them (scratches mame home dvd burns almost as fast as well.. something fast), and if you are meaning to do a real archival, I still believe that dropping a dollar per blank cd balances reliable long lasting media with decent capasity.

        Anyway, most times people are storing way too much for archival... you don't need a full copy of your system in 10 years, you just need the documents and data. As a student I can back up the past three years and probably another 10 years of word docs onto a single CD-R with a redundant image on my school email server... that's a cheap and really sound backup solution.

      • Assuming the tapes survive, the important question is: will you have something that can read the tapes, ten years from now? For instance, I have some 5 1/4" disks that I can read only in the old pentium100 computer that's somewhere in my basement. I'm still looking for a backup solution that's likely to last more than a few years.
        • Any company that backup important material to tape should either make sure they have a working restore-mechanism in place or continue to move the data to newer media every other year...
          That being said, the prices of IDE-harddrives and raid-controllers like these [3ware.com], you might as well keep all your data online. That's a sure way not to let the media become outdated...
          • I agree that a live mirror is perfect for short-term redundancy, given the low cost of IDE storage. However, I don't want to rely on "spinning platters" for long-term data security - aside from the danger of a sudden lightning strike or other electrical disaster taking everything out at once, mechanical media have all too finite lifetimes.
      • I've used tapes for many years (OK, something like 15 or 16) on sites where tape was the backup medium of choice for Big Iron. I can't say that my experience would bear out the reliability of mag tapes as a backup medium. I've run up a lot of instances of I/O errors in the middle of a multi-volume set.

        However, my experience with brand-name CD-RWs on a good quality drive has shown up nearly 100% reliability (only 1 bum disk) over 4 years. FWIW, of course...

      • CD-R is supposed to be good for 5-10 years, but yes, as we all know some media is crappier than others, and it actually has little to do with the brand name, and more to do with what formulation was supposed to be hot that month, year, whatever.

        Magtapes are supposed to be at least that good; better, if you store them in the proverbial cool, dry place. Your real enemy is temperature fluctuation, as it will cause the media to expand and contract. You certainly don't want to get them too hot, though, or operate them too cold (though I doubt you could damage a magtape by getting it very cold, you would have to get it very, very cold...

        • archival cdr's which use non-organic dyes are rated for 50-100 years depending on storage conditions (temp and condensation) Basically the only thing to damage the data is oxidation due to moisture contaminating the reflective layer between the plastic layers.

      • On the other hand, I have videotapes from 5-10 years ago that are barely watchable anymore due to degradation. If an analog signal deteriorates so badly, can you imagine what would happen to the integrity of a digital bitstream?

        I also have yet in several years of burning to encounter a CD-R that worked correctly at first, but later became unreadable.

        Anecdotal evidence is not evidence.
        • That is why digital storage use Reed-Salomon codes, as noted in the write up. These are used in both CD and DVD technologies. In fact, both CD and DVD use double RS coding.
      • What's the longevity of a DVD-R, especially the cheap ones?

        This PDF spec [verbatim.com] on Verbatim's site throws out a figure of 100 years for General Use DVR-R discs. Another PDF I've seen on Pioneer's site says the same thing, but in reference to DVD-R for Authoring discs. We use the latter at the office since Pioneer suggested to us that the Authoring discs are better for archival purposes. The discs are probably 2-3 times the price of General Use discs, but for securing source code, etc. it's worth it.

    • Re:What about speed? (Score:3, Interesting)

      by cdrudge ( 68377 )
      Yes they do pay off. Here's why:

      50/100 GB Tape: $50. Easily unmounted, removed, taken off-site or stored in a safe, etc. Tape changers make backing up to multiple tapes/rotating tapes a cinch. Random seek times for a particular file is slow, but full restores are acceptable.

      Spare 100 GB Hard Drives: $100 If you go with cheaper IDE drives, you probably aren't going to do hot swapping, so you'll have to bring the system down to remove them. SCSI would be more expensive, but would allow hot swapping. NAS is an option though, but slower. Drives are not meant to be carted around, bumped, etc like removable media is. One accidental drop and the entire backup is shot unlike tape or disc based backups. Really fast seek times though.

      CD-Rs: Cheap. $15+ depending on quality/name brand. You can get free at BestBuy or some place after rebate, but usually not worth the hassle for a business. Only holds 700 MB, so be prepared for lots of swapping. Slower then other methods. Plus you now have a stack of 150 CDs for every backup that you do. Can be transported around, but inconvenient. Fast seek times for individual restores.

      DVD-Rs: More expensive then CD-Rs. 20+ for 20 disks for a 100 GB backup. Same things apply to CD-Rs, stack of discs, inconvenient transportation, etc...just less discs.

      This is all hypothetical of course, you circumstance may be different.

      Tape drives were not meant for backing up individual files for restoring those individual files. It was meant for archiving large data file/sets. Tapes offer a large capacity at a decent price with convenience of having all your data in a small durable package unlike what hard drives. Yes there are going to be alternate storage mediums, but there is a convenience/price/durability tradeoffs.

      If you want to be able to seek to a particular file to restore/modify it, go with a CD-R or HD. If you want to backup your system, go with tape.
      • Spare 100 GB Hard Drives: $100 If you go with cheaper IDE drives, you probably aren't going to do hot swapping, so you'll have to bring the system down to remove them. SCSI would be more expensive, but would allow hot swapping. NAS is an option though, but slower. Drives are not meant to be carted around, bumped, etc like removable media is. One accidental drop and the entire backup is shot unlike tape or disc based backups. Really fast seek times though.

        Well this used to be the case.
        Today there are several makers that supply IDE-RAID card that support hot- and coldswapping.

        You can also get drivecages that are suspended from the cabinet so the whole thing can handle transportation without being damaged.
        I even heard a story of somebody having a disk-storage box returned because of damage to the chassis (the sidedoor was cracked in transport) but the disks survived like a charm ;-)

        • I realized that when I wrote that originally. That is why I said you probably aren't going to do hot swapping. I admit that I have not looked into hot swapping IDE drives significantly, but it appears that IDE hot swapping has a tendency to be buggy at best and absolutely dangerous at worst. I wouldn't trust a real server to hardware that "may" work properly.

          Everything adds up extra. I said later down that there were other storage options. Like you pointed out, there are even with actual hard drive. Although adding a IDE hotswap controller and shockmounted drives are going to add significantly to the price...possibly 400-500% over the price of just the drive.
          • A 12-port 3ware escalade costs around $800, you could fit 12 drives @ 240gigs each making it possible to store 2.88 Tera (unformatted of course, what else ;-).

            Each of them drives is gonna set you back ~$200 (easy number) so for $3200 you've got a ~2.5 Tera RAID system... That's pretty cheap considering this is ONLINE data, not offline-backups...
            Buying more disks would offcourse allow you to hotswap and store the disks both offline and offsite, making it a viable backup solution as it takes 200 gigs per medium, and it's fast to backup to and not at least restore from!

      • Spare 100 GB Hard Drives: $100 If you go with cheaper IDE drives, you probably aren't going to do hot swapping, so you'll have to bring the system down to remove them. SCSI would be more expensive, but would allow hot swapping. NAS is an option though, but slower. Drives are not meant to be carted around, bumped, etc like removable media is. One accidental drop and the entire backup is shot unlike tape or disc based backups. Really fast seek times though.

        First of all, the solution to your hot-swapping problem is firewire. You can get an adapter to stick an IDE drive on a 1394 bus. Second of all, hard drives ARE designed to be carried around, this is why the heads park. While the heads are parked it's actually pretty hard to damage a hard drive without dropping it pretty far onto a hard surface, though your mileage may vary on this point.

        Get a IEEE1394 enclosure with a 5.25" drive bay in it, get an IDE hot swap adapter, and hook it all up. When you want to change drives, pull it off the 1394 bus, power down, and swap. All the benefits of SCSI that are relevant here, and none of the disadvantages.

      • Spare 100 GB Hard Drives: $100 If you go with cheaper IDE drives, you probably aren't going to do hot swapping, so you'll have to bring the system down to remove them. SCSI would be more expensive, but would allow hot swapping. NAS is an option though, but slower. Drives are not meant to be carted around, bumped, etc like removable media is. One accidental drop and the entire backup is shot unlike tape or disc based backups. Really fast seek times though.

        I like your post, but you forgot that modern EIDE disks support tunneling the data through another transport. So an ATA hard drive solution does _not_ mandate using an ATA adapter with 2 devices per IRQ in Master/Slave. You can mount an ATA disk using many other kinds of buses and gain hotswappability in the process. ATA bridges for USB, 1394/Firewire, SCSI, and even the parallel port exist and can be bought at Fry's. There are disadvantages, like USB's bad speed and reputation for unreliability, or Firewire's cost, or SCSI's device conflicts. But it can be done.

        I _think_ SATA also supports hot-swapping drives, and there's a standard in the works for using USB for internal peripherals.

        The big downside to ATA drives is storage. Even though head autopark now, they're still more vulnerable to crashes when spun-down. And if a drive mechanism is going to fail, the spin-up after the drive's been in your file cabinet for a year is when its going to happen, maybe due to a little stiction.

      • Re:What about speed? (Score:3, Interesting)

        by evilviper ( 135110 )
        You have good points, but I would add a some more info.

        Hard drives ARE a good medium for backups. Sure, they weigh more, so can't take as much of a drop, but tapes don't have metal casing around them, so hard drives are less vulnerable to electrical/magnetic interference. You could have sheilded cases for tapes, but they are vulnerable while you are handling them. So you really have to choose your poison in that case. Any story of restoring ancient backups always includes the part about all the tapes that just couldn't be read ofter years of sitting in storage closets. Admitedly, that's often due to over-use of the tape, but hard drives are clearly more durable for repeated uses.

        In addition, tapes cost about the same as hard drives for similar capacity, and yet hard drives don't require extra hardware that will set you back several thousands of dollars, so HD are clearly cheaper for the smaller shops.

        In addition, when you spend thousands for your 100GB tape drive, you are stuck. You can't use the 250GB tapes that come out the next year, until you spend several more thousands of dollars for the new drive. Not to mention that hard drives can all be used simultaneously, while you can only use one tape at a time per the number of multi-thousand dollar tape drives you have.
    • Don't forget about OnStream [onstream.com]. They've got a sub $400 IDE solution ($387 on Pricewatch) that does up to 5MB/sec (that's fast) with a 30/60GB capacity (uncompressed vs compressed). The downside is the tapes are costly - about $50/ea.

      For about the same $$, you can get a nice DDS4 DAT drive, which does 20/40GB, and has super-cheap tapes - $17/ea at CDW, probably cheaper elsewhere..

  • Sourceforge has one for OS X (Score:5, Informative)

    by backlonthethird ( 470424 ) on Monday March 03, 2003 @09:08AM (#5423320)
    http://dvbackup.sourceforge.net/

    It works very well. 5 gigs on a tape, 10 if you don't make it redundant.
    • There is also a guy doing a cocoa gui version [geocities.com]. I've tried it and it works pretty well so far (v0.2), but he claims [versiontracker.com] to be releasing v1.0 in April.
    • Sourceforge has one for OS X

      For OS X? The SourceForge page says "Linux", and the dvbackup page makes no mention of anything except "Unix". I just downloaded and it built OK on RH 8.0.

      Firewire isn't available under Linux (AFAIK) with my Asus A7N8X MB (NFORCE2), and I don't yet have an appropriate camcorder anyhow. I may have to get one to check this out. ;-)

      At any rate thanks for the interesting link!

    • Totally off topic, but by using Babelfish to read the original article I managed to end up on this page after a Spanish to English translation. Quite humorous, although I was getting pretty fed up with the authors use of slang by the time I figured it out:



      they open standard: if your to streamer, aah to camcorder dies you dog rescue your dates with any to other one (except PAL/NTSC need to fit), you plows not bound to to special company
      it's to faster streamers than many and it will be dwells comfortable - you dog uses the search-index function to "jump" to to recording

      Wha? The incessant use of the word "dog" was driving me nuts, bringing up memories of that Sean Connery movie from a couple years back. Anyway, back to your regularly scheduled broadcast, dog!

  • Ive seen this before... (Score:5, Interesting)

    by betanerd ( 547811 ) <segatech&email,com> on Monday March 03, 2003 @09:08AM (#5423322) Homepage
    I remember seeing something like this about 4 years ago that used your avreage vcr as a backup medium. Apparently one VHS tape could hold around 1 gig of data, but it took about two hours to get it on there.
  • What's the point? DVDs and CDs are, as far as I can tell, the best way to backup data. I've been backing up data for years, and tape drives are slow, relatively expensive, and provide little protection against time.
    • It may be nice to include applications or Powerpoint presentations on the making of your video or a tutorial with apps to recreate it.

      Further you could include the transitions and extra audio to change it.

      Plus, any multi use is a good use, especially if it were unexpected.

      Recently the Mac Community has seen two very cool apps for Sony T68i phones. They remote control the DVD Player and iTunes + Powerpoint presentations. Sure, there's other remotes that do a better job, but it's still handy. You always have your phone. Just as, some people always have their camcorder and PC nearby.

      I have a digital camera that has an MP3 player built in, it's practically useless, except for the sound effects that I keep on there, in Mp3 format, to add to slideshows. I can edit on other people's computers when travelling.

    • Backup Shelf Life (Score:4, Informative)

      by Christopher Bibbs ( 14 ) on Monday March 03, 2003 @09:29AM (#5423427) Homepage Journal
      The difference is how long the medium will last. If a CD-R degrades after two years it is considered acceptable. Professionals in the video industry expect this stuff to remain perfect for the next 10 years at a minimum. If you need your back-ups to last 6 months until you do your next round of backing up, then CD-Rs are perfect. If you want to back up a system and your data retention policy says you need full archives for the last 7 years to be sure you can comply with an IRS audit, tape is the way to go.
      • If a CD-R degrades after two years it is considered acceptable.

        Interesting. Do you happen to know if DVD-R (or DVD+R) is similar?

        I have a rather keen interest in the answer to this, as whilst typing this in the background Pinnacle Studio is hard at work preparing a DVD image of my wedding video shot on Saturday...

        Cheers,
        Ian

        • Re:Backup Shelf Life (Score:2, Insightful)

          by slittle ( 4150 )
          I have plenty of CD-Rs most definately over 2 years old (pushing 5 for some of them) - no failures yet. They're backups though, which unlike your wedding video are typically never used again (except for me to test of course) and not prone to acts of child/dog.

          I suggest that you burn two DVDs and keep each at a separate location (one at work, one at home for eg). Reburn one (or both, if you want) of them every two or three years.

          I expect my CDs to last up to 10 years, but like I said, they're mere backups that live in a CD folder and are rarely touched. You're talking about your wedding, not ancient pr0n/mp3 archives.
          • I suggest that you burn two DVDs and keep each at a separate location (one at work, one at home for eg). Reburn one (or both, if you want) of them every two or three years.

            Good advice, and advice I'll take.

            I also have the advantage that I run a small-scale webhosting operation. If I copy the DVD image to my server, it will be both an offsite backup and copied to tape by my co-lo hosts on a nightly basis. Paranoid certainly, but as you point out we're talking about data that is crucial to me.

            Cheers,
            Ian

      • Professionals in the video industry expect this stuff to remain perfect for the next 10 years at a minimum


        Professionals in the industry generally know better than to use MiniDV for broadcast-quality recording. Also, the people who made CD-Rs in the beginning claimed a 20+ year shelf life, and nobody should believe them. Nobody seriously claims a data life for a consumer quality videotape, because they're not sold or tested for data-grade applications, but for videotaping, and consumer videotape has a really high "acceptable" loss rate.
        • You may want to lecture them then. DV (which uses the same codecs and technology as MiniDV, just a different physical form factor) is the standard for news footage, documentary work, and indie work.

          Oh, you'll also want to let the journals know they shouldn't be making claims about the use of DV as a long term storage for high quality masters. I mean, they'll be so embarassed when they realize their mistake, but relieved that you set them straight.
          • You may want to lecture them then. DV (which uses the same codecs and technology as MiniDV, just a different physical form factor) is the standard for news footage, documentary work, and indie work.


            MiniDV is a lot different, and it's got to do with the quality of the tape used, which is exactly the part of the system we'd be using for data storage.

            Try looking here [spec-comm.com] for a precis by a company that maintains/repairs hardware for both types of systems, or here [sonyusadvcam.com] if you want a manufacturer's perspective.

            They're not the same. One has much better performance and stability. It's not the consumer grade.
    • If you don't have more than 600 megs, CD-Rs will do, assuming the media doesn't degrade. A DV tape can hold 10 Gb, it would take 15 CD-Rs to hold that much data.
    • What's the point? DVDs and CDs are, as far as I can tell, the best way to backup data

      great! please tell me how to back up to CD or DVD 90 gigabytes of data daily. or better yet the 1.2 Terabyte SQL database I have or how about the 20 terabytes of video files the master video database has here...

      CD and DVD is only a really small scale backup solution for low end home use or special use. Cd and DVD is for content authoring first and foremost.

      Hell it takes 20 CD's to backup my documents directory her at work.. That all the policy and proceedure manuals, company built software manuals, and other documentation.

      I love tape backup and until someone creates a backup solution that is greater than DLT with the same reliability it is the absolutely the ONLY choice in backups that matter.
      • great! please tell me how to back up to CD or DVD 90 gigabytes of data daily. or better yet the 1.2 Terabyte SQL database I have or how about the 20 terabytes of video files the master video database has here...

        Dude, that is a LOT of porn.

    • I have about 160GB of online storage space on my personal workstation. That includes a lot of priceless data. I back it all up to a single AIT tape, which will hold about 200GB per tape compressed. Backup speeds are about 10MB/sec. I have Retrospect back-up the machine overnight, and after the initial back-up, it just has to back-up the changed files, which takes practically no time at all. I rotate tapes on a daily basis.

      Sorry, but for starters, you can't fit 200GB onto a DVD-R. Secondly, I can back-up 200GB to an AIT tape in right around 3 hours. You just can't do that with optical media right now. Not to mention that AIT drives can be found pretty cheap these days if you don't HAVE to have an AIT 3 drive.

      This is why I'll never like optical backup right now. It just can't compete with my AIT drive. I buy archival optical media for long term storage of data, but the workhorse of my back-up efforts for giving me the security of knowing that *everything* is getting backed up every night... I don't know how I could live without AIT.

  • don't bother (Score:3, Insightful)

    by g4dget ( 579145 ) on Monday March 03, 2003 @09:11AM (#5423328)
    Magnetic tapes are a nuisance, and, worse, they don't last very long even if you store them well. Audio and video tapes have additional problems related to their consumer origins.

    With IDE drives down to around $1/G, just go out and buy yourself a bunch of IDE drives for backup--they are a lot more reliable and a lot faster. Or, get a DVD-R: at 4.7G per side, it's not so different from a MiniDV, and they are bound to last longer than a MiniDV backup tape.

    • > With IDE drives down to around $1/G, just go out and buy yourself a bunch of IDE drives for backup--they are a lot more reliable and a lot faster.

      What about when your house burns down, your power supply drops 120VAC on the +5 line of all of the drives, etc? Am I supposed to drop an IDE disk in my safety deposit box occasionally? As far as DVD-R goes, so far the media cost seems kind of high to be running full backups very often. (E.g. nightly, which is easy to do with tape.)

      Note, I'm not endorsing the idea of using video tapes for backup, but using a real tape backup still seems like the best option to me for data you really want to be able to restore.
      • Huh?

        I would certainly hope that you would store your tapes off-site, why not your backup HDs?

        I had nothing but failures on the old (admittedly consumer-grade) tapes I used to use. I back up my personal data to CD now. I really should get better about putting it in the firesafe, though (I think my currents are in my CD binder...on my desk)

        A warning about CD-R backups: I've had CD-Rs fail. This was 7 years ago, but we had some gold-colored discs develop "spots" on the written surface, under the plastic. They made the files generally unrecoverable. The degradation occured in a one year timespan. After that, we kicked around the idea of redundant backups and test-reading our old discs once per month just to make sure we didn't lose anything again. Of course, back then our entire department generated only about 150MB of new data each year.
      • Am I supposed to drop an IDE disk in my safety deposit box occasionally?

        Yes. If the size bothers you, pay a little more and get laptop drive or a bus-powered USB2 drive. There are about half the size of a VHS cassette.

        Or, alternatively, simplify your life, stick the disk into an old computer and do backups over the network (the link doesn't have to be very fast). You can do wireless and put it somewhere reasonably far away from the main computer (garage?), put it at a friend's house, or colocate.

        As far as DVD-R goes, so far the media cost seems kind of high to be running full backups very often.

        Media are down to under $2 per disk for both "R" and "RW" (2-3x that at your local CompUSA). You can use DVD-RW if you want to reuse backups.

    • Re:don't bother (Score:3, Informative)

      by Lumpy ( 12016 )
      that's funny, DLT magnetic tapes have a guarenteed life of 50 years, and we have some 9 track magnetic tapes we found in the basement from the late 70's along with a 9 track tape reader that had an ISA card from the mid 80's and dos drivers (on a 5 1/4 inch floppy) to read that tape last week... it read fine.. the conversion from ebcdic to ascii that was a bitch.

      Yeah, tape doesn't last very long..... I'll go tell these tapes we use and rely on that.
      • that's funny, DLT magnetic tapes have a guarenteed life of 50 years,

        Yeah, right. And if it doesn't last 50 years, the manufacturer will gladly replace the media for you, which is the degree of their liability. And DLT's are at least designed for data and backups, MiniDV are consumer media for video. Get the difference?

        and we have some 9 track magnetic tapes we found in the basement from the late 70's along with a 9 track tape reader that had an ISA card from the mid 80's and dos drivers (on a 5 1/4 inch floppy) to read that tape last week... it read fine..

        I have had 8mm and DAT tapes fail after a few months. Tapes develop a lot of cross-talk between loops if they sit around for too long (you can hear it on audio). The coating used for tapes becomes brittle and just falls off sooner or later.

        Yes, you can get lucky. And 9 track tapes are probably the most robust given that the data density is so low. But it's not something I would trust important backup data to.

      • Given the timeframe, the data density on those tapes was likely 800 bits per inch, almost certainly no more than 1600 bits per inch. A large reel of tape likely held less than a flopp disk. That is a far cry from the data densities used on the tapes in question. It's not hard to understand that an 800 bpi tape, properly stored, might be readable after decades, but you can't apply that to all magnetic media. My experience with magnetic media is far different. I have so many unreadable floppies that a while ago I started copying any driver or other software that came on a floppy and was likely to be needed in the future to CDR (web downloading can't be counted on when even major maker one year can be gone the next). And way too many times I've tried to use various back-up tapes only to find that the tape wasn't viable when you needed it.

        Clearly there are two different almost religious views on this, but many of us have been burned by tape backups too many times to keep putting blind faith in them.

    • "Magnetic tapes are a nuisance, and, worse, they don't last very long even if you store them well. Audio and video tapes have additional problems related to their consumer origins."

      On the other hand, you're thinking about analog signals stored on magnetic tape. Digital signals aren't anywhere near as delicate as analog waveforms (since they can take degredation and still be perfectly readable), so they'll last much longer longer than those VHS tapes you're thinking of (especially the ones you recorded at EP speed).
      • No, I'm thinking of years of painful experience backing up Sun workstations on 8mm and DAT tape and trying to get the data back. I'm also thinking of having old tape coating just crumble off in front of me.

  • The actual software projects used: (Score:4, Informative)

    by sanermind ( 512885 ) on Monday March 03, 2003 @09:12AM (#5423332)
    Are Rsbep - FEC for dvbackup [netic.de]
    and dvbackup [sourceforge.net]
    Using them is relatively straightforward. Actually, it looks like rspep is a generic implementation of reed solomon error correction, which would make it usefull to storing information on any somewhat lossy or error-prone media.

  • Cost effective? (Score:5, Informative)

    by MrMickS ( 568778 ) on Monday March 03, 2003 @09:14AM (#5423343) Homepage Journal
    Let's see:

    You can pick up a DDS-3/4 DAT drive on eBay for under $300. Tapes come in at similar costs to MiniDV (or cheaper). These will backup more than the DV tape and are designed for this sort of use. MiniDV cameras however cost more than this and aren't designed for this sort of use.

    The only reason that I can see for looking at this at all is if people have a MiniDV tape and want to make occasional backups. It could be a risky business for though for two reasons:

    1. You have to explain to your SO can't video the first steps/words of your child because a backup is in progress.
    2. Your SO, searching for a tape, overwrites your only backup with video of the family.

    I remember using the first 8mm backup systems, derived from 8mm video mechs. They were notorious for only being able to restore on the same mech. This would have been fine if the failure rate of the mechs wasn't so high.

    • Re:Cost effective? (Score:2, Interesting)

      by sanermind ( 512885 )
      I recently bought a miniDV camera for only $320 dollars, new. Same price as a drive, faster backup and you can use it as a video camera, too.

    • Re:Cost effective? (Score:5, Interesting)

      by Dot.Com.CEO ( 624226 ) on Monday March 03, 2003 @09:39AM (#5423458)
      I have a miniDV camera. It sits mostly on my desk, since I use it to edit the videos of my family. It is connected on a firewire port (that is up to 400Mbit/sec but on my Sony it is really 100Mb/sec).

      It was designed to be connected to a computer, accepting bi-directional communications with error correction, with a tiny tape that stores up to 1 hour of DV quality (essentially lossless) video, or about 14Gbs.

      If it is already there, if it is already connected to the computer through an extremely fast interface, and if it is so easy to backup data, then what is the problem with backing up your data? That they have a high mechanical failure rate? And what is your proof? 8mm backup tapes?

      This is not intended to replace DATs at the server room, it is just to extend the use of something you already have. This is hacking at its best.

      The only thing that does not excite me is that the tools are not 100% compatible with Windows. Yes, I can compile them with gcc but I want my dialog boxes dammit! :-)

      • Re:Cost effective? (Score:3, Informative)

        by TheRaven64 ( 641858 )
        DV quality (essentially lossless)

        This is not quite true. DV is MJPEG, so each fram is compressed lossily, but there are no intermediate frames constructed just from changes applied to previous frames. This makes it very good for video editing, since each frame can be edited without affecting any other frame, but it is not lossless.

        • He didn't say that it was lossless, but it was essentially lossless.

          i.e. the compressionr ratio is so low that it's uncompressed for all practical purposes.

          Just like it's practically impossible to tell the difference between an SHQ 2048x1536 JPEG from my Olympus C-3000 and a TIFF from the same camera. SHQ JPEGs from that camera equate to using a quality level of 98% or 99% when saving using GIMP or ImageMagick filesize-wise. (Default seems to be 80% or 90% - Even at those quality levels it's hard to see a difference.)
          • The problem is that JPEG uses a DCT-based algorithm, so it can not compress a sharp change from black to white (for example) using a finite amount of space. This is due to the fact that you can not represent a square wave as the sum of a finite number of cosine waves. If it used JPEG2000, or some other DWT based algorithm, this claim of 'essentially lossless' might be true. With JPEG then the claim of 'Good enough in most cases, but hopeless in some' would be more true.
        • DV is NOT MJPEG. DV is DV.

          Also, what I said is that it is essentially lossless. I dare you to find artifacts on any dv movie filmed with a reasonably modern DV camera.

          • DV is NOT MJPEG. DV is DV.

            Okay. While strictly true, DV is DCT based, and so I lumped it in the same category as MJPEG. I could have spent a page explaining the differences, but I didn't bother because they both use more or less the same algorithms, and I doubt anyone would bother to read it if I did.

            I dare you to find artifacts on any dv movie filmed with a reasonably modern DV camera.

            Sure. Film a white background with a black line in the middle. Oh look, on either side of the line there is a hatchy gradient. Why? Because DCT based algorithms can't cope with this kind of source image, which I think was the point I made in my last post...

            • You'd probably find the lens would give up before the format on a consumer camera though.

              We get artifects such as that on our professional DVCAM camera, while the pass through (using Y/C out) is nice and sharp.

              The little MiniDV jobby we have smears before it hatches on fine detail or deliberate attempts to show up the limitations of the format.

              No doubt the DVCAM has an added advantage since the track pitch of the tape is 1.5x wider than DV, although this reduces the recording time by a third.
        • DV isn't exactly JPEG or Motion JPEG, although it's close. It is DCT based, though. MJPEG requires higher data rates to provide equivalent quality.
      • If it is already there, if it is already connected to the computer through an extremely fast interface, and if it is so easy to backup data, then what is the problem with backing up your data? That they have a high mechanical failure rate? And what is your proof? 8mm backup tapes?

        I don't see an issue with using your camera for occasional backups, it's your camera :) I was pointing out there are cheap solutions to this issue that may be more suitable.

        WRT mechanical failure. I have no proof for miniDV. Apply a little bit of logic though. The mech in a consumer level device is primarily designed for record and playback on a streaming basis. Backup tends to be like this, restore less so. If you are using your device for regular backups you are likely to use it more than just as a video device. This will wear everything more. The more you use it the more chance of it failing.

        Back in the early 90s video 8 was released. At the same time this was seen as a more cost effective medium for backups than QIC (Quarter Inch Cartridge) tapes and so a modified version of this mech was used for data backup. The problem was that the tolerances for data and video are different and the drives had problems with alignment. This led to problems reading tapes written on one drive not being readable on other drives. It could be the same model from the same manufacturer but it just wouldn't read. Once DAT became more widely available 8mm lingered on a while (it had higher capacity) but was gradually replaced.

    • I wonder why camcorders don't use DDS technology. DDS are 4mm tapes, instead of DV's 8mm. Even with this smaller size, a DDS4 tape can hold 20 Gb uncompressed, 40 Gb with truly lossless compression, versus 10 Gb with lossy compression in DVs.


      Now imagine a DDS-based camcorder, with a built-in mpeg4 encoder. It would fit in the palm of your hand, and be able to record 50 hours of video in one tape which is about half the size of a DV. That would be possible with current technology, using OEM parts that are available right now.

  • Preventing data loss using striping (Score:4, Interesting)

    by arvindn ( 542080 ) on Monday March 03, 2003 @09:20AM (#5423379) Homepage Journal
    One of the major problems with using tapes for storing data seems to be dropped frames. A tiny spot of dirt or liquid may lead to several thousand consecutive bytes being missed, leading to data loss. Error correction schemes can cope with only so-many consecutive bytes getting lost. However, it is possible to cure much larger number of consecutive bytes lost using striping (as used in RAID). Basically, it ensures that logically consecutive bytes are physically spaced apart. Thus, a dust particle would now lead to a few bytes being lost in each of a number of frames, which can be corrected. A minor drawback of this is that decoding would no longer be frame-by-frame but blocks of several frames at once, amounting to several megabytes perhaps, but this I don't think this is a serious issue.
    • I don't read spanish very well, but the article mentioned using Reed-Solomon encoding which does just that.
      It mixes up large blocks of data and then does ECC on them, so if a large block gets dropped due to an error, it actually contains a small number of bytes from many other block, each protected by a separate ECC.
      This is the same type of error correction used on CD's (CDROMs uses 3 layers of ECC).

  • It's an interesting poing, though (Score:3, Interesting)

    by lingqi ( 577227 ) on Monday March 03, 2003 @09:20AM (#5423381) Journal
    Have anybody ever held a VHS tape and simply marvel at how much data it can (theoretically) hold? I mean, DVD is 4.7G (probably more) for a movie, compressed as much as 40:1. That means that each VHS tape can theoretically hold close to 1/5 of a TERABYTE. taking off for some redundency, let's say 10X, I should still get 15G out of a 2 dollar tape.

    I remember there was people who made "VCR backup solutions," too. Anybody knows what happened to them?

    before anybody gets too negative, remember I am talking 10X redundancy for 15G. Heck it's sure a lot cheaper than DVD-RW right now, so from price/GB point of view, I think it still kicks DVD's butt, even if higher redundancy was needed.
    • Uh, and what do you base these wild assumptions on?

      First off, a DVD is 9.4G/side - most current DVDs are dual-layer. Rewritable DVD's are still only single layer.

      Next, you've deeply, vastly, absurdly overestimated the quality of VHS. Record at best quality possible on VHS and it's still less than half the line count of DVD, and we won't even go into chroma and lumiscence loss.

      The one estimate I've seen for VHS backups is 2.7GB... certainly not that bad, but it has all the drawbacks of tape storage and requires a rather hokey control system (IR blaster out of the PC).
    • I recall seeing these reviewed a few years back. They let you fit about 2GB on a tape. The problem is that VHS videos degrade very easily. If you watch a video and look carefully, you can see that the colours aren't always consistent, and that there are small flecks. Each one of these could result in massive damage to data stored. The conclusion that came out of the review was that they were basically write-only devices. You could store data on them, but getting it off was unlikely to happen.
    • I'm not talking about those cheesy "Use your VCR as backup" devices, those can only be described one way: Cheap hacks. They are limited by all the circuitry in the VCR that mangles the input signal. (The AGC circuitry, for example. Note that this is the same circuitry that allows Macrovision to work.)

      I'm talking about D-VHS. (Such as the JVC HM-D30000U, and 1-2 other DVHS decks in existence.) These are designed from the ground up to record digitally, and given the bitrates specified and the recording times, it comes out to something like 40-50 gigs per DVHS tape. Not sure how well "standard" VHS tapes work, they're probably exactly the same except for a cost premium for the DHVS-specific tapes.

      Current limitations:
      a) DVHS decks record MPEG-2 transport streams, not DV video streams. Which means dvbackup won't work. DVHStool for Windows is about halfway there - But it only will send prerecorded or premade transport streams. The VCR might be unhappy if the TS doesn't have a real MPEG stream in there, but one could embed a low-bitrate 480i stream and fill the rest of the transport stream with data. (MPEG-2 Transport Streams as used in HDTV broadcasts allow for arbitrary data. For example, a Fox station broadcasting 480p with no subchannels is padding their 19.2 MBit/sec stream quite a bit.)
      b) Current DVHS decks are rare and the ones that do exist don't have the best reliability record.
    • "Have anybody ever held a VHS tape and simply marvel at how much data it can (theoretically) hold? I mean, DVD is 4.7G (probably more) for a movie, compressed as much as 40:1. That means that each VHS tape can theoretically hold close to 1/5 of a TERABYTE."

      According to JVC, digital VHS can hold up to 4 hours of HDTV resolution or 24 hours of SDTV.
  • Some info I found a while back whie looking into the same topic:
    Rsbep [netic.de] (Reed-Solomon and Burst Error Protection).
    Looks like the current version is 0.0.4 with the caveat "it still gives occasional bit-errors, especially in Long-Play mode or with damaged tapes." The code is doubly FEC'ing data, which seems odd, but supposedly/should increases the integrity of the data.

    Just some info...

  • MiniDV is most suited for use in Video or Audio recording and playback devices. They cannot compete in reliability, speed, or efficiency with media such as CD/DVD, or even your ordinary harddisk (which is not so ordinary anymore)... With it being a tape, it doesn't offer efficient Random Access, which in today's world is a needed feature even for backup purposes. When you need to restore your data in a hurry because your business depends on it, your miniDV will leave you wanting...

  • (...)this should let you backup 13GB in a cheap MiniDV tape in a safe manner, and restore it all later.

    Hmm...

    • Is this really needed?

      ... and restore it all later.

      Yes. There were a lot of backup devices around a few years back which let you back-up data to VHS for under $50, but the one thing they all had in common was that most of the time you couldn't restore it later, since VHS tapes are not very reliable.

  • Finally... (Score:2, Funny)

    by roalt ( 534265 )

    Finally a way to store all my mpeg's...

  • In a pinch, sure... (Score:3, Interesting)

    by Spoing ( 152917 ) on Monday March 03, 2003 @09:56AM (#5423542) Homepage
    ...though I wouldn't trust it on a regular basis.

    As pointed out in the previous comments, DV cameras are not designed for lossless data handling, they are designed for video. Video is lossy, and while the tape and recorders are capable of error free capture and transfer, the emphasis is on making it good enough.

    The cameras will dump the tapes at normal recorded video speed. If an error occurs in transfer (record or dump), the camera will keep going. If either computer in the chain is slow, you have to increase the error correction -- decreasing the available storage -- to handle those errors.

    So, you end up getting better quality tapes and/or loosing capacity and speed.

    Additionally, to use the DV you have to have a firewire port on the computer or drag along a card. If you have the time to do the prep work to put in a firewire card, you might want to consider getting a firewire hard drive -- and move from 5-10 gigs a tape to 60-120 gigs a drive. The speed is much higher too.

    With that much forsight, you may as well pick a case that has USB 2.0 on it to cover systems that don't have firewire already. That's what I did, and it's worked out well. As can be expected, USB 1.x works too, though very slowly.

  • Might be a bad translation, but... (Score:4, Insightful)

    by oneiros27 ( 46144 ) on Monday March 03, 2003 @10:00AM (#5423577) Homepage
    It might just be the broken english from using a translator, but I'm just not getting how he's dealing with the issues that have been brought up time [slashdot.org] and time again [slashdot.org].

    But let's be even more realistic -- hard drives are almost $1/gig these days. Even if he has worked out the issues that are mentioned (lossy media...I really don't get the new discussion how lossy is lossless), the media deteriorates with age, seek time, etc, the cost savings just aren't there. The only advantage that I can see on this [assuming it does work well], is the physical size of the tapes, and so, you'll be able to pack more data into a fire safe.

    Oh...and don't forget... at 13G/tape, you might be looking at 10+ tapes to back up some drives these days....It might be a cool hack, but changing tapes sucks...I'd still rather just mirror to another drive that I can remove, as the random access dramatically improves recovery time.
    • Why should I shell out $150 (CDN) for an additional hard drive to create a duplicate of what is already in the computer?

      If I have a mini-dv camera already, I can back up my drive for $10. I just saved $140 bucks and can store the backup somewhere outside my computer, safe for such things as fire, flood, or theft.

      Hacking at its best, taking what you have and making it more useful.

      Michael
      • You might be able to fit a drive onto a 13G tape, but I have systems at home that have over 350G of drive space on 'em. [and even with them at 50% allocated right now, that's still a bunch of tape swapping, and definately more than $10 worth of tapes].

    • Oh...and don't forget... at 13G/tape, you might be looking at 10+ tapes to back up some drives these days....It might be a cool hack, but changing tapes sucks...I'd still rather just mirror to another drive that I can remove, as the random access dramatically improves recovery time.

      But if you're a home user, not backing up an IT department or your business machines, then you don't need to back the entire drive. Just the changes. And for a single PC, even with broadband, a day's worth of work isn't going to fill an entire 100GB hard disk, unless you're doing video production or audio editing with ProTools. Then your back in the "business" category of backups, and should buy a solution indexed to the cost to recreate lost data. Never backup $1000 worth of data on a $35 device unless you know your going to be able to recover it.

      This isn't really a reply to your post per se, but slashdot in general. Everytime one of these "backup" related stories hits slashdot, it's always the same litany. "How do I backup my 136 GB hard disk? To another hard disk?" No. The answer is "You don't." Or "If you have to back up the WHOLE disk, you're doing it wrong."

      1) Secure your computer.

      If you tighten the permissions on your PC so that non-root/administrator can't write files wherever they please, this simplies backups greatly. You shouldn't be logged in as a privelidged user anyway except for maintenance. Your backup software can help you here, making a catalog of changed files to backup. If you see changes in /usr, /bin, or even Program Files or WINNT, you should figure out what those changes are and whether they need to be backed up at all. You may need to tighten permissions again or reconfigure the program to write its logs or scratch files somewhere less dangerous, like the temp directory.

      2) Don't back up worthless files.

      Browser cache, history, temp files, print spool files, etc etc. Just exclude them from the backup script. You're probably saving yourself from future prosecution by not backing up any way, and some of the files get corrupted and are good to purge. If you don't backup worthless junk, there's less to backup.

      3) Backup in sets to multiple media.

      If you've done the above, then you can use algebra to make backups. If you've already backed up the OS and apps, and your permissions and backup scripts confirm for you that there are no interesting changes in those directories to backup... then you don't need to back them up! As long as you have a duplicate of each backup tape/cd/floppy (copy 1 of 2, copy 2 of 2) and you store them in different places, you're good. I've been running on incremental backups for months.

      On a home machine, with good security and a good list of junk files to exclude, a days changes are at most 1 meg, but only if I download a large file. And that includes my wife's stuff too. With an aggressive backup scheme, it really doesn't matter how big the disk is but how much change in data from day to day you have to back up. And it gives you a good opportunity to learn about your system by keeping tabs on this.

      With that kind of strategy, MiniDV looks actually pretty good. Just use 2 mDVs with the same backups on each, maybe "Winter 2003 copy 1 of 2" and "copy 2 of 2". That will hold 13GB of _changes_ to your system, which is quite alot, and might actually hold a year's worth for some people. With the reduced cost, it might be worth a look.

      • I know you weren't replying specifically to me, but I just wanted to mention issues with #3 --

        Yes, you can back up only incremental changes, however, if you're backing up to something like tape, you need to use a good backup program to maintain indexes of the tapes...

        otherwise, you're going to be going through every last tape you have to find that file that someone accidentally deleted.

        [for me, recovery time is key, as if I want something back, I want it now....but of course, all people are different, so some others may see the added cost/effort now to not be worth it]

  • Longevity of media (Score:2, Insightful)

    by s.surfer ( 37335 )
    If people treated their tapes the same way that most CD's are treated, ie. throw them in a pile in
    the corner, then most tapes would have a very short lifespan.

    Most companies that I have worked for that used tapes send them off site,
    and put them into a controlled environment (humidity, temp, etc).

  • I used to think this was a good idea... (Score:3, Interesting)

    by Skevos Mavros ( 460902 ) on Monday March 03, 2003 @11:49AM (#5424322) Homepage
    But waiting over an hour to back up "only" 13Gig? By the time this sort of software is ready for prime time (ie, it has a half-decent GUI), DVD blanks will be dirt cheap, as will really fast DVD burners. I'd rather send the same data to around 3 DVD disks. Especially since the stuff I like to back up is the kind of data where I often need to retrieve just a few small(ish) files from the lot. Near-instant random-access is something I've grown accustomed to.

    Then again, for system backups which allow a complete restore of the whole lot, this could be handy, I guess. Especially since the "tape drives" (cameras) have lots of other uses. And I guess the speed of most DV cameras' rewind and fast-forward means that this would allow almost-speedy random-access.

    It seems like this idea of backing up data to DV tape has been around for a while - but there's still no "prosumer"-ready application out there (yeah yeah, go write one yourself - but I'm not a programmer).
  • Try searching EBay for Sony's AIT drives: I immediately found some as low as $200. They use 8mm cartridges derived from Sony's 8mm video technology. I've used the 35GB and 50GB (native) models in earnest for a few years ago, and while they're not as fast as DLT, they're much better in every way than DAT/DDS, in performance and reliability. The tapes are far less likely to get squashed by a mug of coffee, too.

    On the other hand, I've been testing a HP Ultrium-2 drive, which costs a bomb-and-a-half, but is giving real speeds of 16MB/sec backing up, and close to the claimed 30MB/sec on restore. It won't stream unless your subsystem can handle 15MB/sec sustained, which is not trivial in the PC world. When it does, though, you can almost hear the slurping...

  • Okay, the tapes are cheap, but the DV cameras are expensive. Furthermore, they generally have highly-compact (read failure-prone) read-write equipment.

    This is a cool hack, but not very practical for most people.
  • Some guy did this for Mac OS X months ago. I discovered it sometime in January (here [pedrovera.com] is what I posted at my site with my experience with it).

    The way this one is set, you can put either 10 GB of data without error correction, or 5GB with error correction. It is rudimentary but in principle works and it is simple to implement. I guess it took me less than an hour to figure out how to make it work with my JVC miniDV cam.
  • Are we talking about the types of cassettes used in MiniDV camcorders/DVcams? If we are, I'm not sure where the author is getting "cheap" medium from: a 60/90 minidv cartridge costs anywhere from $7-13.
    • I forgot to mention that the cost of a 'reader' for these MiniDV would be at least $300 - the cost of a refurbished miniDV camera. Unless there are simply ieee1394 miniDV readers, which I've not seen yet myself.

Slashdot Top Deals

Our OS who art in CPU, UNIX be thy name. Thy programs run, thy syscalls done, In kernel as it is in user!

Close