Please create an account to participate in the Slashdot moderation system

 


Forgot your password?
Close
typodupeerror
×
The Internet Data Storage

Online, Inexpensive and Secure Data Storage? 87

Posted by Cliff from the tired-of-digital-media dept.
ThePolkapunk asks: "After years of suffering through floppy demagnetizations, hard drive crashes, CDR bit rot and the click of death, I've become fed up with having to take care of my own backups. Does anyone know of a reasonably inexpensive, secure data storage facility accessible online that can store all of my important files with enough redundancy for me to feel safe?"
This discussion has been archived. No new comments can be posted.

Online, Inexpensive and Secure Data Storage? More

Online, Inexpensive and Secure Data Storage?

Comments Filter:

  • Gmail (Score:2, Informative)

    by Anonymous Coward
    http://www.gmail.com/ [gmail.com]
    • I also use GMail for some light duty backup purposes, and to further help the paranoia I use http://www.truecrypt.org/ [truecrypt.org] to put smallish (1-5MB) encrypted partitions up there for sensitive data. So far they have scaled with additional storage faster than my demand for storage, so I am quite happy with it so far. Just store the executable for TrueCrypt in your account as well and you are set for life.

      Very cool.
      • For life? Or until gmail notices that you're using their service to store files and disables it. Or until they close, or require you to pay. Or until they decide to make a change that breaks gmailfs. gmail is great and all and it's fine to use for "light duty" as you say but calling it "for life" is a real stretch. Anyone that uses it as the primary or sole means of backup is really asking for it though.

  • Simple... (Score:5, Funny)

    by Anonymous Coward on Tuesday June 14, 2005 @02:23PM (#12815578)
    Compress everything, encrypt it, and share it on emule as "OSX_X86.iso.

    For extra redundancy, post an obscure blog about it, and then submit that blog to /.

  • USB/Firewire drives (Score:3, Informative)

    by andrews ( 12425 ) on Tuesday June 14, 2005 @02:23PM (#12815581)
    I've recently switched from DLT to external USB drives. They're cheaper, more reliable and easier to restore from than tape.

    • Re:USB/Firewire drives (Score:5, Informative)

      by gregmac ( 629064 ) on Tuesday June 14, 2005 @02:43PM (#12815838) Homepage
      I've recently switched from DLT to external USB drives. They're cheaper, more reliable and easier to restore from than tape.

      I've been doing backups this way for a few months now. It's also easier to backup. Plug in the usb drive, run rsync, unplug and take offsite. I have a script I run in cron (or manually) that tries to mount it first, and only if it suceeds, runs rsync.

      I've been meaning to install another drive at my house to backup a couple servers (just haven't gotten around to it yet), but that'll be similar.. rsync over ssl, run in cron every night.

      The one thing this doesn't address that tapes usually do is keeping multiple backups. I was doing this for a while - where I had a directory for I think monday, wednesday and friday - but then our storage space increased and I couldn't fit this on my backup drive anymore. It would just be a matter of buying a bigger drive, or getting another couple enclosures and drives, but I haven't done it yet.

      I was playing with using rdiff-backup to make differential backups of just our file shares, but due to lack of a usable web interface I haven't bothered (I don't want to spend time selectively restoring backups for people.. if they can't do it themselves I want no part ;) ). I did start working on one, but there are too many things higher on my priority list.

      • Re:USB/Firewire drives (Score:2, Informative)

        by Anonymous Coward
        I plug in and mount my USB disk once a day and run this. I could probably use a for to make it shorter, but... meh.
        more information [mikerubel.org]

        #!/bin/sh
        #SRCDIR requires a trailing /
        SRCDIR=/home/acoward/
        DSTDIR=/media/usb/acowar d/rsync/

        cd $DSTDIR
        if [ -d daily.7 ]; then
        rm -rf daily.7
        fi
        if [ -d daily.6 ]; then
        mv daily.6 daily.7
        fi
        if [ -d daily.5 ]; then
        mv daily.5 daily.6
        fi
        if [ -d daily.4 ]; then
        mv daily.4 daily.5
        fi
        if [ -d daily.3 ]; then
        mv daily.3 daily.4
        fi
        if [ -d daily.2 ]; then
        mv dail

        • If you are going to keep a week of rsync'd backups, reduce your script to one line by employing the date command and it's formatting option:

          rsync -a --progress --stats --delete --link-dest=../Mon $SRCDIR `date +%a`/

          Run "date +%a" to see what it does.

        • Great script; thanks for the rsync line. BTW, if you wanted to use a for loop, here's one way:

          for i in $(seq 6 -1 1) ; do
          if [ -d daily.${i} ] ; then
          mv -f daily.${i} daily.$(( $i + 1 ))
          fi
          done

          Note that you don't really need to delete the daily.7 dir first; the mv -f should take care of that. Also, I think the $(( ... )) operator might only apply to bash, so you may have to change the !/bin/sh to !/bin/bash at the top of the script.

          Another suggestion is to use a version control system to stor

        • You're missing out on a really neat trick that can radically reduce the amount of disk space needed for multiple copies of your data:

          instead of just using "mkdir daily.0" do a "cp -al daily.1 daily.0" before rsync:

          this creates a copy of the previous days backup directory tree, with all files as hardlinks to the previous days data.
          Meaning: only space for the directory entries is used, actual file content is just referenced. running rsync over this hardlinked tree will result in just the changed files being

        • I use rdiff-backup [nongnu.org] for actual incremental backups. rdiff-backup is based on librsync and works amazingly well. There are a few spites such as a lack of checksums and non-numerical user ID's which can be dangerous when restoring a full system, but overall it is very impressive.

          If you're solely using (external) HD's for your backups, you'll like DAR (Disk ARchive) [linux.free.fr] even better than rsync/rdiff-backup.

      • You need rsnapshot [rsnapshot.org] and a *nix box.

        I have up to 12 months worth of backups, including hourly snapshots of the previous 24 hours, always available to all the users online. Tapes do not have to be hunted down and mounted. The users can easily go browse their own snapshots and get the files they need when they have an oops. The snapshots are maintained on a different machine than the source data, so an individual machine can melt into a lump of slag with the loss of no more than one hour of data.

        Since it uses

    • Re:USB/Firewire drives (Score:3, Insightful)

      by OctaneZ ( 73357 )
      The whole reason people prefer tape for backup over hard drives, is that you seperate the stoarge medium from the recording device. You therefore have an easier time recovering if something fouls one or the other.
      • i guess thats because tape drives & tapes foul up so often ?
        seriously, anybody using tape for backup is asking for trouble
        they suck unless you can afford a huge robotic tape thingamjig where even if half the tapes are bad you still have something
        tapes are dying...
    • Hmm you got that post in quick but missed the point.

      Heres my suggestion for online inexpensive secure data storage. Encrypt your backups and share them on a p2p service as hot.slutz.divx.avi.
  • You can throw all your stuff in my server if you'd like. Make sure it's unencrypted, though, I'll take care of all that for you.

    Seriously, though, I store all my backups myself, because it's important enough to me to back it up but I'm not a corporation that can pay someone to back it up for me. Assuming you're like me, you only have one option.

    • Re:My server (Score:3, Informative)

      by rogabean ( 741411 )
      I tend to actually take this a step forward in my thinking...

      If it's important enough for me to feel I need to back it up, it's important enough I would not trust anyone else besides me backing it up.

      Couple of external drives works great for me. Important stuff is backed up twice (once to each drive).

      YMMV
    • You can throw all your stuff in my server if you'd like.

      Now there's an idea. A group of friends could mirror each other's data (encrypted if they don't trust each other).

      If three friends got together to do this, each would have two offsite backups.

      N friends would have N-1 offsite backups.

  • Backups?? (Score:4, Funny)

    by rogabean ( 741411 ) on Tuesday June 14, 2005 @02:26PM (#12815624)
    Sage Advice:

    "Only wimps use tape backup: real men just upload their important stuff on ftp, and let the rest of the world mirror it ;)" - Linus Torvalds
  • A decent RAID setup is easy and fairly inexpensive these days.

    • Re:RAID (Score:4, Insightful)

      by HawkingMattress ( 588824 ) on Tuesday June 14, 2005 @02:52PM (#12815959)
      Not sure if that's what you mean or not, But RAID is *not* a replacement for backup. If you don't pay attention and delete a file, a backup should be able to give you that file back. A RAID setup will ensure that the file gets deleted on all the mirrors...
      On the other hand if you use an external RAID server to store backups on yep it's not bad. But i'd say that it would be even better to just have a regular ide disk to make backups on, and a remote location you can upload your backups to... RAID is good, but what happens if you PSU goes crazy and kills all the disk in the array (happened to me :( ) ? Or if the physical location the server is in catches on fire ?
      So if you can have a place to upload to and it's doable for the amount of data you need to transfer, it'll probably be much cheaper, and much more secure than a RAID array.
      • All valid concerns.

        But if you're that paranoid (and after losing a whole RAID array in one go, I don't blame you), you're going to be pretty hardcore and nuanced about your backup strategies.

        But, for fairly pedestrian usage, not to mention cost and ease of maintenance (Dead HD? just slap in another one), RAID can't be beat.

        I'm also running under the assumption that you have a PCs plus RAID storage setup, so if the RAID goes to shit like yours did, there's a set of data still on the PCs, right?

        Bo

        • RAID is a great way of ensuring that your data is nice and consistent. If you're running RAID and you overwrite a file, delete one by accident etc etc then what you end up with is a perfectly redundant and consistent state: with your file deleted. RAID is for availability, backup is for safety.

        • WTF (Score:1)

          by lorcha ( 464930 )
          Problem:
          > rm -f ~/important_file
          > # Oh shit, I didn't mean to do that!

          Please tell me how RAID would fix this problem. I'm waiting...

          Waiting...

          Still waiting....

          Oh, that's right. RAID doesn't fucking solve the fucking problem. That's what fucking backups are fucking for.

          Fuckin' A, you're dense.

          • Boy, you people are clumsy with your data...almost all of the complaints I'm getting are of this kind. I don't know about you, but it's been a good few years since I 'accidentally' erased, or lost a file. You people need to be more careful what you rm, rf or carelessly drop.

            • it's been a good few years since I 'accidentally' erased, or lost a file.

              Bet you really wanted it back, too. What if you mess up a DELETE or UPDATE statement in your database? Or bork a configuration file and want to see what it used to look like?

              What if your power goes out in the middle of writing a file to disk and the file gets corrupted? What if you lose multiple disks on your RAID at the same time because they were from the same manufacturing run? What if multiple disks are damaged while t

      • Not sure if that's what you mean or not, But RAID is *not* a replacement for backup. If you don't pay attention and delete a file, a backup should be able to give you that file back.

        That's why you should also take periodic snapshots of your data.

        I use Linux LVM over four-disk RAID-5 for most of my data, and LVM over four-disk RAID-1 for my most important data. I have a cron job that creates daily LVM snapshot volumes and removes any snapshot more than a week old, unless it was taken on a Monday, and

    • I'm not sure how putting bug spray on my hard drive will keep it from crashing.

  • As usual.. (Score:4, Insightful)

    by rylin ( 688457 ) on Tuesday June 14, 2005 @02:27PM (#12815635)
    As usual, someone thinks they can have all three.
    Here's a hint: Pick two.

    • Er, why not use GnuPG (free) + 2 Cheap Webstorage/hosts.

      Its cheap, its secure, and since the data is on two physically different machines at two different locations it should be reliable.

  • How 'bout Gmail? (Score:4, Interesting)

    by xmas2003 ( 739875 ) * on Tuesday June 14, 2005 @02:28PM (#12815652) Homepage
    GmailFS - The Google File System [slashdot.org] as discussed on Slashdot ...
    • 10 mb limit on any given file...
      • Really, they should allow you to register your 5 google accounts, drag a 10gb file on the folder, and then split/span the file into all those accounts, which can be done through one root account, (if sent mail doesnt add to storage...) anyway, keep sent mail can be turned off.

        Then, when you login, it pops the 5 accounts one at a time, collates the file system, then caches.

        Since you only have to update when you make a change, it wont be slow.

        Google has some neato throttling on file dl's, but if you copy t
      • Uh... partition files in 10MB chunks with your favorite zip/rar/compression scheme?
  • At least as important as the others is stability. You don't want a site that winks out for a few hours because four guys decided to upload 20 meg .mov files at the same time. You also don't want a fly-by-night dot-bomb that is fast, meets legitimate security requirements, and is free.....but it inexplicably vanishes 13 days after you discover and start using it.
  • I would like to know too. I have 80gigs to backup. I currently use a friends machine, but it would be nice to pay about $20/month and not have to maintain it mayself.
    • > I have 80gigs to backup. I currently use a friends machine, but it would be nice to pay about $20/month and not have to maintain it mayself.

      Get another 80 Gig hard drive. They go for about $0.40/Gig with rebates (see www.bensbargains.net). Put it inside your machine or in an external USB enclosure.

      Get Norton Ghost, or if you don't want to pay for Norton Ghost, make a Knoppix CD for free and use 'ntfsclone'.

      Make exact images of your hard drive once a week. It may not be "online" but it couldn't get m

    • Important issue, many seem to forget. (Score:1, Insightful)

      by Anonymous Coward
      Many people seem to forget the single biggest issue with online backups. That is, performance. People seem to forget that it will take nearly ten days to upload 80GB even on a 1Mbps broadband connection.

      What happens when your hard drive fails and you need to restore? Can you really wait for ten days or more while the restore is downloaded? In my opinion, if you can't do a full restore in four hours or less, then you need a faster backup/restore solution.

      Online backups aren't really very useful unless you

  • Easy (Score:4, Interesting)

    by g1zmo ( 315166 ) on Tuesday June 14, 2005 @02:38PM (#12815765) Homepage
    my_backup_14-6-2005@yahoo.com
    my_backup_15-6-2005@yahoo.com
    my_backup_16-6-2005@yahoo.com
    my_backup_17-6-2005@yahoo.com
    my_backup_18-6-2005@yahoo.com
    my_backup_19-6-2005@yahoo.com
    ad infinitum...

  • Backups (Score:4, Informative)

    by mknewman ( 557587 ) * on Tuesday June 14, 2005 @02:39PM (#12815783)
    You can try these guys: http://www.box.net/ [box.net] Marc
    • Looks OK, but the problem with some of these online backup systems is that they use a web interface, and don't seem to offer FTP. Looks like cheap web-hosting accounts (with FTP) are a better option.

      Simon.

  • EVault.com (Score:5, Informative)

    by gothzilla ( 676407 ) on Tuesday June 14, 2005 @02:40PM (#12815787)
    Check out http://www.evault.com/ [evault.com]
    Very reasonably priced and they maintain offsite backups of their backups.
    One thing to keep in mind is transfer speed. I would have used them but I have 40gigs of data to back up and it would take too long to send daily.
    • If you can use rsync then only the changes are transferred (so after the first big dump additional backups are small). I use a combination of Unison and rsync to maintain files between my laptop and desktop and between my desktop and backup.
      • Their site says their special software does what amounts to an rsync, however the parent poster was probably referring to the initial transfer of all the existing files.

  • Consider Magneto-Optical (Score:3, Interesting)

    by VernonNemitz ( 581327 ) on Tuesday June 14, 2005 @02:43PM (#12815840) Journal
    This technology is based on a scientific method used to detect the orientation of the Earth's magnetic field from millions of years ago. That qualifies as great data retention in my book! The drives (3.5in) run $200-$300 depending on source and capacity (current max is 2.3Gb per approx-$20 removable disk). 5.25in drives are also avaialable in capacities up to 9Gb or so, but also are much more expensive.
    • 2.3Gb per $20 disk on a $200-300 drive?

      Have you seen how cheap hard drives are these days?? For that much money you could buy several 40Gb drives and mirror them all.
      • The relevant question is, "How much data are you going to put on those many-gig hard drives? The average person will be able to to manage with a not-too-expensive total number of MO disks. Hard drives are for storing Operating Systems and Applications, not data. Because no hard drive has the guaranteed-40-years storage lifetime of MO. And so far, every new generation of MO drive has been backward compatible with ALL the previous generations of MO disks. If your MO drive dies, you get another and all
  • I wouldn't call it cheap, but I doubt you will find much cheaper than 10 gb for $5/mo [sagonet.com].

  • Distributed Internet Backup System [berkeley.edu]: I backup your stuff, you backup mine. All encrypted.
  • I just put all my files on my web server with an index.html that says it's an open source project.

    That way somebody will mirror it and others will provide torrents.

    Information wants to be free!

  • Yahoo offers online file storage with a set ammount for free, and a fee-based structure for buying more space. Upload is through the web, and it also allows you to set up public folders for files you need to distribute.

    Barring that, you can use an external hard-drive. I use a firewire storage enclosure with a 250 GB drive to back up my notebook, and to provide extra storage for video editing.
  • My boss uses a system I don't entirely approve of, but it works regardless. I work at a sound studio, and he does the following: Current projects are on his main computer's drives, backed up to a 1 TB LaCie nightly. Once a week, he brings in a firewire drive from home and backs up recently finished projects. In essance, he dumps projects onto drives as they are finished.

    When the terabyte drive gets full, he stores it in a fire-safe cabinet in the studio, so he is able to access it within a few minutes (plu
    • Yeah, against my recommendation we have a 1TB LaCie "Bigger Disk" as well as 2 320GB LaCie "Big Disks" which also happen to be used for audio files. All of these devices use multiple drives and RAID0. RAID0 is not RAID, there is no redundnacy and your data is actually at *greater* risk because you lose your data if any single drive fails.

      The Bigger disk and one of the Big Disks both failed in the same month. Fortunately, the Bigger Disk files also existed elsewhere but replacing the contents of the Big Dis

  • See all the current losses in backups (Score:3, Insightful)

    by MerlynEmrys67 ( 583469 ) on Tuesday June 14, 2005 @03:30PM (#12816441)
    Here are companies paying big BIG bucks for these three (well, probably not online - they'll ship tapes) and you still have tapes lost/stolen off of the UPS truck.

    My advice - make sure you SECURE your data before it leaves the house... ie. run AES over the whole thing. Depending on your parania level - either keep the key locally, or with a DIFFERENT backup provider so that there would have to be a collusion between the two vendors to get your data.

    Another solution would be to encrypt the AES key for each backup with your public key - then all you have to do is keep the private key private. Is it small enough to keep on a pair (or more) USB dongles - or again, back it up with a second vendor.

    End result - most backup vendors provide physical security, so it is up to you to provide true security for when their physical systems are broken into.


  • The really ideal solution would be a p2p backup network. Each client has their own encryption keys, and agrees that for every X kbytes of local data they need backed up, they're willing to store 3X worth of data from others. Your data is encrypted with your key before being sent out to the p2p network, which is designed to keep a minimum of 3 remote copies of your data out there - if less than 3 copies are detected for any decent amount of time, it just makes more copies on other machines. When you need
  • Here's my regime:

    Materials: 1 Linux server, 1 winxp desktop, 2 removable HD cartridges.

    The desktop has a removable hard drive cradle [makeitsimple.com].

    1. Every week I plug in one of the HD's to the XP machine and mount it onto the linux box via samba.
    2. I tar my entire linux box, piped through GNU Privacy Guard [gnupg.org] (to protect against unauthorized access) and 'split' (to avoid the 2GB file limit) and store it on the HD in date-stamped directory.
    3. I store the removable HD offsite.

    The backup drive is much larger then the c

  • http://www.yournetpod.com/ [yournetpod.com]
  • My vote is with USB drives. They're inexpensive enough that you can ghost a drive image onto a couple and, say, store one in a safety deposit box and another with a trusted friend/relative in another state.
  • Jerry Pournelle's been doing this for years. Get a fire-proof safe. Get a Post Office Box. Make CDs of all your important data as often as you feel necessary. Daily. Weekly. Bi-weekly. Whatever. Make like 5 at any given time. Put one in your safe. One in your PO Box. Keep one at your office. Keep some in a trusted friend's garage. Bury some in a chest in your backyard. Keep some at your mom's place. Or your kid's place.

    The odds of all the CDs going bad or all the places blowing up at the same time are pre

  • ...then upload it to an FTP server and let everyone else mirror it.
  • Secure. Online. Inexpensive.

    Pick two.

  • A good thing to remember about backups (and one that no one in this thread has yet brought up) is that most of your data isn't that important. Sure, you've got a 200 gigabyte hard drive that's full to bursting, but how much of that is MP3s and how much is your really important, totally irreplacable data? You know, your finances, your emails, even your vacation pictures?

    A quick check of my own hard drive gives me about a gig for my /home directory on the Linux side and about 3.5 gigs for the My Documents and Application Data directories on the Windows side; meanwhile, my Media directory has about 110 gigs in it - all ripped from other media, and thus replacable. Even the 4.5 gig figure could probably be pared down considerably, if I wanted to put the effort into it - especially in Windows, applications tend to leave temporary files everywhere. Although I wouldn't be surprised to hear that many people here have far more data than I, I would also be very surprised to hear many non-business users say that the majority of the information on their hard drives is totally irreplacable - that is, there are more than a few non-professionals who have more space devoted to their own work than they do programs, ripped media, and the like.

    Why is this important? It makes it a lot easier to back up your data if you can isolate the important bits. Instead of a fairly expensive solution involving removable hard drives, it's probably possible to fit everything on a much cheaper archival-grade DVD or two - or store everything on a remote backup server for a lot less money. In fact, with only a few gigs of mostly static data to consider, an FTP or SSH-based reciprocal backup arrangement with a friend could work well.

    The point here is that, while backing up every single bit on your hard disk is an excellent idea, and will certainly get you back up and running quickly, it can be much more feasable to simply back up the small subset of truly irreplacable data. While it's not as good as a full system backup, it's certainly better than the full system backup you never implemented or used because it was such a hassle. While this kind of system might be unacceptably lax in the buisiness world, it is certainly better than what many home users, including many Slashdotters, are using right now.

    • You raise a lot of good points, but I'd like to mention that backing up media files offsite isn't a bad idea.

      Homeowners and renters insurance may limit how much they'll pay to replace lost media. Even then, you have to contend with finding the media: CDs go out of print, Disney stops selling certain movies for a time... even the companies themselves go out of business. (I've got CDs from labels that don't exist anymore, and were never re-released.)

      It's all a matter of priorities. :)
  • When I bought my first DVD burner, I specifically chose one that was DVD-RAM capable. Little did I know that the media are 5-10x the cost of DVD-RW. So, what is the verdict on DVD-RAM, are they really engineered to last decades and 10's of thousands of rewrites? If so, how do they do it?
    • way OT, this is a new askslash topic

    • Take one of your DVD-RAM disks, and look at the underside. Notice the pretty pattern? It's not just there to make them look good! That's hard sectoring; you never ever need to low-level format a DVD-RAM disk.

      The DVD-RAM disks do actually last a lot longer than the normal RW disks. I don't know how much longer though, as none of mine have ever died (unlike the RWs). While it may not be so widespread in the western world, I have heard that DVD-RAM is far more common in the Far East.

      I also heard somew

  • Streamload [streamload.com] offers a free online storage, retrieval and storage service. It takes only a few seconds to sign up for an account and you don't have to pay to store your stuff (they give you unlimited storage with no additional charges), you only pay a nominal charge for the amount of stuff you download back to yourself above 100 MB. Even the subscription plans are cheap (starting at $5 a month). I've used the service for about a year now and it works very well though using just my browser. I would recommen
  • http://www.backup-connect.nl/?pid=6 [backup-connect.nl]

    They offer online backup services in many parts of europe for reasonable prices, and you can go 30 days back (up to 180, if you pay more.) My company uses it (6 people) and we sleep quite soundly : )

    - Jynx
  • Selling online remote data storage internally is the toughest proposition.This is the quintessential problem,right? Even if you do find a good remote backup service provider,(this remote backup whitepaper [data-recov...eviews.com] is a good starting point to find remote backup services providers)you are going to find it difficult to convince your bosses who are always paranoid about possible security violations

Slashdot Top Deals

Our OS who art in CPU, UNIX be thy name. Thy programs run, thy syscalls done, In kernel as it is in user!

Close