Mass Storage Leaves Microchips in the Dust

Roland Piquepaille writes "This article from Wired Magazine looks at storage with a new angle. 'Right now I am sitting in front of a whirring 60-gigabyte hard disk that cost less than $100. Do the math: If back then 10 megabytes cost $1,000, then 60 gigabytes would have cost x, where x = $6,000,000 and "back then" = 18 years ago. I'm sitting in front of $6,000,000 worth of mass storage, measured at mid-1980s prices. We have Moore's law for microprocessors. But who's coined a law for hard disks? In mass storage we have seen a 60,000-fold fall in price -- more than a dozen times the force of Moore's law.' DeLong also looks at a non-distant future when a $100 mass storage device will hold a full terabyte. He also thinks that with disk space becoming cheaper and cheaper, we'll be tempted to archive everything about ourselves, including pictures and videos. This is in fact the goal of the Gordon's Bell project, MyLifeBits. You can learn more about the MyLifeBits project by reading this NewsFactor Network article. Check this column for more details."

Price

[larry2k]

If back then 10 megabytes cost $1,000, then 60 gigabytes would have cost x, where x = $6,000,000 and "back then" = 18 years ago

No only the price, the size of the drives. 18 years ago a 40 Mb HD has the size of a toaster...

Yep

[jackal!]

18 years ago a 40 Mb HD has the size of a toaster...

Yep. Generated the same amount of heat, too.

Planes should be made out of recycled black boxes

[NanoGator]

" 'Right now I am sitting in front of a whirring 60-gigabyte hard disk that cost less than $100. Do the math: If back then 10 megabytes cost $1,000, then 60 gigabytes would have cost x, where x = $6,000,000 and "back then" = 18 years ago. I'm sitting in front of $6,000,000 worth of mass storage, measured at mid-1980s prices. We have Moore's law for microprocessors. But who's coined a law for hard disks? In mass storage we have seen a 60,000-fold fall in price --"

You mean that all this time we could have had much faster computers just by using magnetic media?

Re:Planes should be made out of recycled black box

[Uber Banker]

Yeah exactly. There is no moore's law... just a regular increase which has maintained at a steadyly increasing rate.

There is no 'law'...

Re:Planes should be made out of recycled black box

[NanoGator]

"Whirring is the key word. Hard disks have gotten bigger. But they haven't gotten faster. They haven't gotten quieter. "

Techhnically they have. 10,000 RPM hard drives haven't been around forever. They first showed up in 1996.

Re:Planes should be made out of recycled black box

[NanoGator]

"The point was that they haven't gotten faster in proportion to the increase in size. "

Ah I see, that's true.

Not that surprising, though. The mechanical arm inside of the drive has its limitations. I wonder what it'd take to replace it with a magnetic field sorta like what TV's use to fire energy at the phosphors on the tube. I wonder if a disc (maybe optical disc?) could be read that way. Seems like you could dramatically increase it's read speed that way.

Blah I'm sure there's a serious issue that I

Re:Planes should be made out of recycled black box

[Gossy]

Whirring is the key word. Hard disks have gotten bigger .... They haven't gotten quieter.

You haven't used a Seagate Barracuda IV have you?

Trust me, they have got a lot quieter than the 80s.

Re:Planes should be made out of recycled black box

[Jason1729]

Whirring is the key word. Hard disks have gotten bigger. But they haven't gotten faster. They haven't gotten quieter.

But they have gotten warmer. My computer is starting to take over the job of my heater in the winter.

Jason
ProfQuotes [profquotes.com]

but when it comes to harddrive SPEED

[Anonymous Coward]

we have jon's law.

as in the toilet.

note to harddrive manufacturers: i'm not impressed. i'm still waiting on my data to "move around".

this is news?

[ender_wiggins]

This is just common knoledge. And if he paid 100$ for a 60gig drive, he got screwed! Thats why there cheap, cause dumbasses pay too much for drive, and the manufacs pass the savings on to ME.

iPods for Example

[tbmaddux]

Look at the iPod... it's been out for 27 months and its capacity is up to 30GB from 5GB, or 6x. That is, on average, a doubling in size every 9 months!

In general the problem is that while capacities have lept up, the rate at which we can read/write to those drives has not kept pace. It's not so bad for the iPod in particular, but at some point it's going to be a real problem for desktops and laptops, assuming our appetite for capacity grows as the capacity does.

Re:iPods for Example

[Elwood P Dowd]

Not really. For the price of that 10MB disk, back in 1985, now you can buy an IDE RAID setup that could saturate your PCI bus.

Re:iPods for Example

[Anonymous Coward]

Look at the iPod... it's been out for 27 months and its capacity is up to 30GB from 5GB, or 6x. That is, on average, a doubling in size every 9 months!

Please step away from the crack pipe. 2^(27/9) = 8. However, the storage capacity has only grown 6x. Perhaps you meant to say 2^(27/10.45) = 6. I.e. doubing in size every 10.5 months? :) Or if you're really insistent on the 9 months part, you could say it has increased by 81.7% every 9 months.

Re:iPods for Example

[shaitand]

It's already a major problem, the hard drive is a huge bottleneck in the system... granted it makes more difference on windows machines where the paging is constant but the problem is there on any system.

Why would there be a problem?

[Kjella]

In general the problem is that while capacities have lept up, the rate at which we can read/write to those drives has not kept pace. It's not so bad for the iPod in particular, but at some point it's going to be a real problem for desktops and laptops, assuming our appetite for capacity grows as the capacity does.

I've not seen any indication that we are using anywhere near the hard disk speed. Movies and music have typically gotten less bandwidth-intensive with better compression, not more. It is more li

Re:iPods for Example

[barc0001]

Sorry to nitpick, but your math is incorrect.
By your 9 month doubling:

0 months - 5GB
9 months - 5GB x 2 = 10GB
18 months - 10GB x 2 = 20GB
27 months - 20GB x 2 = 40GB

So to make that true, the iPod would have to be shipping with a 40GB drive right now..

Re:iPods for Example

[theLOUDroom]

In general the problem is that while capacities have lept up, the rate at which we can read/write to those drives has not kept pace.

No, not really. The sustained transfer rate of HDDs has been steadily improving. It's obvious if you think about it. The drive stays the same size. The density goes up. The disc spins at the same rate or faster. Therefore, more data goes by the heads per unit time. This means trasfer rate will also increase. Specifically, this means that the transfer rate will scale linearly with the density of the disk.

What hasn't improved at the same rate as density is seek times. Seek times have always been the killer for mechanical storage mechanisms. They have to move something around and they have to obey Newton's laws.

In order for seek times to improve at the same rate as the rest of the drive impoves, we would need improvements in materials science and motor design which far exceeded those that increased density.

The other neat thing to think about is the spinning discs inside the HDD. Both those impovements I just mentioned might also allow you to spin the platters faster. This means that you could actually increase the transfer rate of your drive as well.

The immediate problem I can see is that moving something back and forth doesn't scale as nicely as storage density. Here's an example:
Say you've got something that you need to get from point A to B. Say you can do it in 1 microsecond. If you want to be able to do it in 1/2 microsecond, you need 4x more force. This means you need a motor with 4x more force, and a material that's 4x stiffer and 4x stronger.

Even if materials science, and motor designs were improving at a rate comparable to "Moore's Law", seek times wouldn't. Some things just don't scale the way we would like them to. Batteries are a good example.

Correction: You need a material that has 4x better strength to weight and stiffness to weight ratios.

It's also worth considering that you have to burn 4x more energy to move something from A to B twice as fast. Power dissipation in CPUs scales linearly with clock speed.

Re:iPods for Example

[jedidiah]

That only really works well for you if you access your disk drives like a magnetic tape (sequentially). Otherwise, you're going to be seeking all over the place and accessing data in a manner that may be too random to exploit disk caching mechanisms.

If you want to get to a particular block on the disk (rather than what happens to be under the read hit) HD seek times still blow.

Re:iPods for Example

[Ed Avis]

At some stage the density may be high enough that you can store everything on a single cylinder and not need to move the head at all. That would also make the drives a bit cheaper to manufacture.

Of course I expect moving-head drives to still dominate because people would rather have a larger capacity, even if accessing most of that capacity requires a few milliseconds of seek time.

Recording Everything?

[Bob Vila's Hammer]

I don't want to see Gordon Bell's "lifebits"

Price?

[gazuga]

In mass storage we have seen a 60,000-fold fall in price -- more than a dozen times the force of Moore's law

Moore's law says nothing about price though. If you are going to compare hard disks to processors in the same general terms using Moore's law, shouldn't you compare increase in storage size to increase in processing power?

Re:Price?

[Kjella]

In mass storage we have seen a 60,000-fold fall in price -- more than a dozen times the force of Moore's law

Moore's law says nothing about price though. If you are going to compare hard disks to processors in the same general terms using Moore's law, shouldn't you compare increase in storage size to increase in processing power?

Well, is Moore's law about:
a) Transistor count
b) Clock speed
c) Processing power
d) Speed per dollar
e) Anything to do with computers that looks like an exponential curve?

Personally, this "moorification" of everything is driving me nuts. It must be the most (ab)used law in computing, with no scientific basis except "Uh this fits well with an exponential regression"

Kjella

Re:Price?

[The_K4]

Um.....Moore's law says it will double every 18 months. Last time I checked that WAS exponential (not linear). It always looks linear on the graphs because people use logrithmic graph paper. The HDs just have a steeper linear slope on the log paper. :)

Bloat will kill the increase in storage available

[jgaynor]

Bloat will kill the increase in storage available - one way or the other. It'll be a 3gig version of word, or windows movie maker that will only save in raw, non-compressed video. Anything to drive the market. We've seen it with processor speeds, if HD prices keep dropping I'm sure well see it with storage as well.

Come on, is XP is SO far ahead of NT 4 that it requires 4x the ram? Of course not. But what MS reccomends, PC manufacturers will have to yield to.

It's digital media, not apps

[benwaggoner]

Nah, not really. There is only so big an app can get in terms of code, and it's way, way less than a small HD can do. For consumers, who are using more than 25% or so for their drive, it's likely digital media that's doing it. MP3 files, and especially video. Textures and videos in games. Tutorial files for media apps. That kind of stuff.

Crack open your average 20 MB MacOS X .app, and you'll likely find less than 25% of the total file is being used for application code. The rest is multilingual help, grap

Re:Bloat will kill the increase in storage availab

[Telastyn]

Haven't we already?

Say the old Gold Box D&D games (1 floppy, maybe 2) compared to Baldur's gate (4+ years old, 4 cds) for example...

Re:Bloat will kill the increase in storage availab

[tbmaddux]

3gig version of word...

No, no, no, NO.

It will be a 3gig version of IIS, .Net, or whatever. The extra 2.9gigs are bundled data so you can buffer overrun yourself.

Re:Bloat will kill the increase in storage availab

[indiigo]

Ever try and run NT4 on a 300 mhz system with ~only~ 64M ram? Runs pretty damn well.

Then add all the SP's and IE6, IT...SLOWS...TO...A...CRAWL...

Re:Bloat will kill the increase in storage availab

[shaitand]

I don't think that's what he's talking about at all. First of all XP doesn't run as well as NT 4 did, xp starts faster, runs much much slower. This is what programmers fondly refer to as the bloat effect, it's where you see excel hold a hidden flight simulator. It's where you look at code and see redudent loops and time delays in what should be a time critical application. It's where you see developers who use as many prewritten libraries as possible because they are too lazy to spend 5 minutes writing

Re:Bloat will kill the increase in storage availab

[Teancom]

A library that saves you 3hrs on a project that takes a year is not good. A library that saves you 3 days on the same project... not good. a week, still not good. A month... now we are starting to consider using the existing library.

While it's nice that you took the time to rant about how much better of a programmer that you are then everyone else (the whole "If I didn't code it, it's crap" attitude really shines through), I think your scale is a bit off.

Lets say a library saves you a week. Now, lets say that like more people you use at least 4 libraries. Now, you've saved a month. A *month*, at which point you say you'll start to "consider" using external libraries. Well, I'm underpaid, but lets say you hired me to do this. By shaving a month off, you've saved over $3500 in my salary alone. And that's assuming that I (or anyone) could fully implement, *debug*, and "finish", a given complicated lib in 1 week. Great! Now, I quit, because I'm underpaid, and my replacement comes in. Now, I write good, well documented stuff, but it's not industry standard. So my replacement can't just sit down and pick up where I left off, but has to learn how *I* decided to implement libfoo. But it turns out that he's a lot like you, and thus 'he didn't write it, so it's crap'. And then *he* spends a month throwing away my stuff, and redoing it all. And on, and on, and on. There's a *reason* that things like Boost and Roguewave and Qt and Gtk and glib exist. And until you figure that out, you're doomed to be 1/10th as productive as you could be. Or, assuming that (as you claim) you've polished your libs to perfection and the productivity is there, I pity whomever has to take over your code. No, actually, I just pity you.

Re:Bloat will kill the increase in storage availab

[Teancom]

I believe what has made code so bloated these days, is people believing it's more important to produce more faster and fix it later than do it right the first time.

I don't believe that I said anything about shipping broken code. Maybe I did. Maybe you could point out where. Or is your point (again!), that if you didn't write it, it can't possibly be Right(tm)?

And in your example nobody has saved $3500 on your salary alone. Every 3rd party lib you add for the sole purpose of saving time at the moment

Re:Bloat will kill the increase in storage availab

[Erwos]

"Basically if programmers still gave a damn like they did when writting code for C64's we wouldn't have alot of these issues. Nowdays they would rather churn out crap so long as it's better than some of the other crap they've seen."

If you're trying to tell me we should go back to the days of non-portable assembly, I think I'm going to cry. Yes, people should write tighter code, but trying to make believe that we should write code just like in the "good old days" is ignoring years upon years of advancement in the field of computer science.

And, also, look at what they were doing back in the days of the C64, and look at what they're doing now. You really do need more code to do more. Trying to tell me that they had 6kb executables with the C64 and then telling me our 6mb ones are bloated is ludicrous.

-Erwos

This really helps but in perspective...

[reverendG]

This really helps to put into perspective the ass-whipping I got when I installed Wing Commander 2 on my Dad's new hard drive.

"THAT 800 MB HARD DRIVE COST ME 500 DOLLARS, AND THAT GAME TAKES 72 MB?!!!"

"But dad, in 15 years that will only be 25 cents of space!"

Re:This really helps but in perspective...

[Stephen VanDahm]

I remember the 486 with a 255 MB hard drive we had when I was in middle school. My Dad was pissed because DOS and Windows 3.1 took up nearly 70 MB of precious storage space when the version of DOS that came with our old Tandy 1000 EX fit on a single 5.25 inch floppy diskette. Dad was even more pissed when I filled the drive with games, WAV files, and pictures. Back then, I was excited whenever I was able to free up another 750 kilobytes of disk space. Then there were the hard decisions...is removing Rise of the Triad, my all-time favorite game, worth freeing over 20 megabytes?

When I started college I bought a Pentium with a 4 gigabyte hard drive. Unlimited storage space! Well, until a friend showed me this awesome new program called "Winamp."

To this day, I'm very frugal with disk space. My home directory resides on a 60 gigabyte drive split into 3 20 gigabyte partitions, and I'm only using 17% of one partition right now.

Spintronics

[metatruk]

A lot of the developments that have made disks so high capacity came from spintronics research. Here is a link to an article on Scientific American about how it works: http://www.sciam.com/article.cfm?articleID=0007A73 5-759A-1CDD-B4A8809EC588EEDF [sciam.com]

DeLong's Law

[Hadlock]

So DeLong wants a computer term/law named after him. Whee. Next story?

-2, offtopic/in a pissy mood.

Only a terabyte?

[gricholson75]

He's predicting only a tearbyte for a $100 in 2012. Right now desktop drives are about a dollar a GB. So, he's predicting about a 10 fold increase in the next 9 YEARS!!! What have we seen in the last 9 years, about 100 fold increase?

Faster than moore's law

[Muerte23]

A few months ago i figured out [slashdot.org] that hard drives have doubled in size every 12 months as opposed to processor power doubling every 18 months.

If that rate continues, some day hard drives will become so large that processesors will not have the power to process it all....

I will know that day has arrived when the length of my winamp playlist rolls over into negative integers. :)

Muerte

Re:Faster than moore's law

[unicron]

Dude, seriously, what the fuck are you talking about? Process it all? Your cpu doesn't "process it all" now. If talks with what it needs do. I'm also pretty damn confused as to what you mean by negative integers? Hopefully that was some weak attempt at a buffer-overflow joke or a stack dump or something because the logical part of my brain thinks you meant you say "I will have so many mp3's that the number system itself will reset" and then I would be forced to clown on you.

Re:Faster than moore's law

[nomel]

By "process it all", he probably means being able to address the area on the disk (think extremes).

By go over into negative integers, integers are an allocated space in memory that holds a number...if the number is bigger than the allocated space, what does it do!? 11111111 + 1 = 00000000 (keeping 8 bits of data). Look up signed integers. Since it's just binary...how can you represent a negative number? Well, you can't directly, you do it with little tricks that everyone agrees on. Look it up...you obviously need to.

Re:Faster than moore's law

[dvdeug]

Dude, seriously, what the fuck are you talking about?

Seriously, dude, it might be nice to know what your talking about and speak English, instead of using phrases like "to clown on you".

Your cpu doesn't "process it all" now. If talks with what it needs do.

But the more and more data you have, the more likely you are to try and handle large quantities. Search every text file on your system, or merely scan and process a file at 600 DPI instead of 300.

I'm also pretty damn confused as to what you mean by negative integers? Hopefully that was some weak attempt at a buffer-overflow joke or a stack dump or something because the logical part of my brain

The logical part of your brain obviously never studied computers very much. In assembly, if you continue adding to a signed integer value, it will overflow to negative. In 16 bits, 32767 + 1 = -32768, IIRC. If you program in C or Fortran or any other language that doesn't check overflow, the same thing will happen. I've seen reports that I had transfered -2 GB this session, because the program overflowed at 4 GB. Same principle.

Re:Faster than moore's law

[Muerte23]

:)

thanks for explaining my joke. it's sad when people flame from ignorance...

muerte

Now on channel 1443 - Bob's Life

[rdewald]

I have to admit that the notion that it is now techincally possible to mpeg-1 every moment of one's existence is a staggering one.

If you accept that Blogs satisfy some previously underestimated human desire for self-expression, think of what might happen if one could clip a web cam to one's collar, wear a storage device on one's waist and synch that with an online VidLog every night like a Palm Pilot?

I am going out back to sit among the dandelions.

Re:Now on channel 1443 - Bob's Life

[Reality Master 101]

I have to admit that the notion that it is now techincally possible to mpeg-1 every moment of one's existence is a staggering one.

Eh? My digital camera does 30 seconds in about 5 meg (rather poor quality, I might add). 80 years of life would be around 411,000 gigabytes. Not exactly practical yet.

It's freaky

[Apreche]

An interesting feature of OpenNap is that it tells you exactly how many MB of files are out there for your downloading pleasure. I used to be blasted away at the large number. Sometimes I could get it up to 1 or 2 pedabytes.

A terabyte is 1000 gigs. You can get a terabyte of storage today for $1000 dollars. One dollar per gig. It's insane. Soon it will be a dollar a terabyte. We wont need things like divx anymore. We'll be looking for ways to increase the quality of our recording devices so that the video, image and audio files will take up more space. Nothing else really requires a large amount of storage.

The one limited is network speed. Sure, if I've got enough room for a collection of 2 gigabyte raw avi movies, that's great. But if I can't get enough speed to download them quickly it will suck.

Storage aint worth crap if you dont' got enough stuff to fill it.

Remember the days when DOS games would ask questions like this

minimum install (if you're low on space)- 50MB
standard install (reccommended)- 100MB
big install (runs faster)- 250MB
CRAZY INSTALL (no cd required!) - 500MB!!!

those were the days...

Re:It's freaky

[Dr Caleb]

CRAZY INSTALL (no cd required!) - 500MB!!!

CD? What was this CD doodadthingie? I remember playing Space Quest on 3 1/2"ers and having to swap floppies! (before I got my Seagate 10M RLL drives.)

Re:It's freaky

[rdewald]

I remember upgrading my 10MB MFM HD in my IBM XT to a RLL-encoded 32MB model and wondering if I would every use that much storge space the rest of my life. Honestly.

Even better, I remember when we got the extra 16k of RAM in the Heathkit I was hacking and wondering what I would ever want it to do that would require that much memory.

I'm old (42).

Re:It's freaky

[Rob Parkhill]

The one limited is network speed. Sure, if I've got enough room for a collection of 2 gigabyte raw avi movies, that's great. But if I can't get enough speed to download them quickly it will suck.

This has always bugged me... back in 1992, I had a 25MHz CPU, 8MB of RAM, a 660MB hard drive, 2.88MB floppies, and a 28.8 modem.

In 2003, I have a 2.2HGz CPU (88 times faster), 1024MB of RAM (128 times more), a 120GB hard drive (180 times more), 700MB CD-RWs (243 times bigger) yet only a 1Mbit (on a really good day!) network connection (about 35 times faster, no matter what the cable company claims.) And that's as fast as it has been for about 5 years now.

Where oh were is my 5Mbit cable modem? Heck, some poor bastards are still stuck using 56k modems...

It seems that network connections ony get faster in big bursts. In 1997, I had a 56k modem. In 1998, I had a 1Mbit DSL line. Maybe in 2008, I'll get fibre to my house.

Re:It's freaky

[gwernol]

Over time, the growth in capacity is awesome.

Last month I was going overseas for a vacation so I decided to buy a new CompactFlash card for my digital camera. For about $100 my camera now has 365,000,000 times more memory than my first personal computer [u-net.com] had. That's insane.

I love living in these times.

Re:It's freaky

[Kjella]

For about $100 my camera now has 365,000,000 times more memory than my first personal computer [u-net.com] had. That's insane.

Somehow it freaked me out when my processor started having more L1 cache than my first computer (A Commodore 64).

Also I remember seeing an old 10mb ESDI hdd the size of a shoebox. When I got my first real PC with a 20mb 3,5" disk I felt that was like "wow, how much better than this can it get". Us young and naive people. Now I'm "old" (read: 24) and jealous of the people growing u

different constants

[ndevice]

looked at another way, hard drive capacities have just been doubling faster than processor speeds.

If 10MB back then cost $1k then 1MB cost $100, so we just do the 60G/1M and get a 60,000 time increase in storage capacity for the same price. Doubling times would then be log(2)60k = 15.9 or so, or about once every 1.1 years over 18 years. Contrast this with moore's law which states that processor speeds double every 1.5 years.

The downside is that access times have tracked closer to a linear function.

Re:different constants

[El Cubano]

The downside is that access times have tracked closer to a linear function.

Too bad it is practically a horizontal line.

Same investment

[Shadow Wrought]

So for the same investment of $100 over the years you could have had anywhere from 1meg to 1 terraflop of crap on your computer. Same low price, a million times the crap!

Yeah! Technology really is making our lives better!

Hey, I'm doing this, too!

[Anonymous Coward]

Actually, I started a project to do just this for myself about 7 years ago. I'm 28 now and it's taken me until NOW to catch up (and I'm only about 80-85% caught up.... still have plenty of baby pictures to scan). I just spent last weekend sifting through old documents about yours truly that my parents has stored away. I now have a ton of interesting data about myself on tap, including every report card I had until college. :)

The original motivation for this project for me was the realization that my gen

doesn't seem much faster than Moore's law

[brett42]

From the given numbers, it looks like storage capacity doubles every 14 months instead of the 18 it takes for chips.

from $6000000 to $100 requires about 16 doubling periods:

100*2^x=6000000
2^x=60000
xln2=ln(60000)
x=15 .872

~16 doubling periods in 18 years is about 14 months per period. Of course, a small change with exponential growth causes major changes as time goes by, but a four month shorter doubling period doesn't seem very significant.

Re:doesn't seem much faster than Moore's law

[shaitand]

every 3 years that's a full year gain... in 30 years that means hd's will be 40yrs ahead of chips... you don't find 10yrs to be significant when we are talking about technology??? The problem with drives isn't storage capacity... they push capacity because they've failed to solve the real problem with drives... the real problem is speed. HDD's aren't really ahead of chips, they are behind.. they should be doubling in speed every 14months not capacity... I'd settle for a 10gig drive that was 14 times as f

I'd rather have small-mass storage devices

[saskboy]

It would be much better if we could combine this growth in the industry, into producing CF cards that can hold 2+ GB, and give us mass storage on small microchips.

Full record

[Pentagram]

In a few years time, when storage is cheap enough, I'm going to have a camera permanently strapped to my head (think better minaturisation + wireless tech) recording my life full-time. At moments of boredom I'll be able to relive any part of my life.

How much storage? Say, 500Mb/1hour (better compression as well, hopefully) * 24 * 365 ~= 4.4Tb/year. Doesn't seem that far away...

Rule of Thumb

[Doodhwala]

The common rule of thumb is that storage capacity (density) increases at approximately 60% a year. There was an exeception around a few years ago when it was increasing by around 100% a year but experts feel that we are settling back at 60% again.

what's the point?

[glwtta]

Don't we have this every couple of months now?

"Have you noticed, hard drives are just so huge now! Oh my gosh, pretty soon we are all going to archive our entire lives! Whoa, I am a visionary!"

Re:what's the point?

[Unregistered]

Whoa, I am a visionary!"
Along with10 people above you and probobly 2x as many below. /. is just full of visionaries today.

The above was humor. Or if you're british humour.

The key is knowing when your life is half over

[perydell]

Record everything. Once your life is half over you need to cue up the recording and start watching what happened in the first half of your life. Then when that is over you drop dead.

Storage isn't the challenge...

[sterno]

The challenge is no longer whether you can store everything, it is whether you'll be able to find it later when you need it.

Yes its all really great...

[irn_bru]

Until you have to defrag the bugger...

A "dozen times the force?" No.

[pclminion]

If prices fell by 60000 times over 18 years, then we're looking for the solution of the equation: x^18 = 60000, which implies x = 1.84 years = about 22 months. So prices fall in half every 22 months (assuming it's exponential). Moore's law says transistor density doubles every 18 months. It's not that much different, although it does make a huge difference over time.

Whoops.

[pclminion]

Ok, I must be tired. We're trying to solve: 0.5^x=1/60000 implies x = 15.87 doubling periods. Spread this over 18 years, 18/15.87 = 1.13 years per doubling (actually, halving). So prices drop in half every 13 months, not every 22 months. Urgh..

in perspective

[u19925]

the disk storage has reduced cost faster than moore's law in recent times. however, the overall rate is not significantly higher than microchips. in mid 80's, 8 kbytes ram used to cost $50. today, you can get 512 MB for the same price. that is 64K tims price reduction which is same as for hard drive. also, the ram price reduced faster than harddrive from 1956 (hard drive introduction year) till mid-80s. in 1956, 5 Meg hard drives were available. in mid-80s, this figure barely jumped to couple of gigs. that

Digital solipsism.

[mikeophile]

Am I live, or am I Memorex?

"Moore's Law" and What Moore Actually Said

[fm6]

We have Moore's law for microprocessors. But who's coined a law for hard disks?

Actually, all Moore did was predict that the complexity of integrated circuits would increase exponentially, without a corresponding increase in cost. (Here's the original paper [intel.com].) This is usually cited as "Moore's Law" and cast something like, "The number of transistors on an average chip will double every 18 months." Which is more than Moore actually said, but a logical inference.

You hear people refer to the assumption that electronics will keep getting cheaper and and cheaper as "Moore's Law". Nit-pickers hate this, insisting that "Moore's Law" only refers to the number of transistors on a chip. But even casting Moore's predictions as a "Law" goes beyond what Moore actually said. So it makes just as much sense (or just as little) to speak of the whole economic trend as "Moore's Law". After all, the fact that transitor logic keeps getting cheaper and cheaper isn't obvious to most people. The resulting collapse in the cost of computing and electronics is.

Nothing to solve the problem of data impermenance

[ispq]

Creating bigger hard disks does nothing to solve the problem of reading data from existing storage devices. As time goes on our society stores more and more information without any real plan on how to ensure that the information we're collecting will be accessible in the future. Every year we lose more and more precious data to the deterioration of media as well as the loss of the equipment to read the remaining media.

Related to space and time?

[PseudoThink]

While hard disk sizes have grown much faster than CPU speeds, I don't think hard disk access times/bandwidth have grown as fast. Perhaps it's a property of the universe, that it's easier to shrink technology in space (making things smaller) than in time (making things quicker)? Maybe, maybe not (femtosecond lasers, etc.)...

Capacity isn't everything

[0rbit4l]

It's great and all that the time to double capacity of mass storage devices is less than the time to double 'capacity' (usually measured in transistors) of modern microprocessors, but it's fallacious to suggest that mass storage is doing 'better' overall. In fact, you can't really say which one's 'better' since they're so different in nature.

Moore's law is largely due to manufacturing improvements in which the feature size of transistors keeps becoming smaller, such that you can get (approximately) twice as many transistors in the same amount of space. (yes, yes, I know, die sizes keep growing, but not nearly at the pace at which transistors shrink.) The tricky part here is that this shrinking has generally been coupled with ramping up frequency. Increasing the capacity of a disk has no such benefit due to the fact that mechanical parts (disk heads, spinning platters) are the overwhelming determining factors for performance. Hence, the gap between processor performance and disk performance is being exacerbated - we can only make a disk spin & heads move so fast.

It's an interesting comparitive trend to notice (between processor performance growth & disk capacity to see the effect on the overall system), but you can't really compare the way disks have improved with the way microprocessors have.

On exponential growth, and media idiots

[freeweed]

A dozen times what processors have increased!!!! WOW!!!! Holy crap are hard drives ever fast!!!

Oh wait, Moore's Law (which has nothing to do with hard drives, but I'll bite) says that things DOUBLE in a certain period of time. Hmm, a dozen times is less than 2^4. Even using the old standard of 18 months (for a while there it was 12), that's less than 4 doublings. 4x18 months = 6 years.

So, let's see. These numbers go back about 20 years, and the difference is less than 6. 6/20 = 30%. Wow, hard drives

Processors = reliable, hard drives != reliable

[SuperBanana]

But who's coined a law for hard disks?

Except that processors don't just give up the ship randomly(well, except in VERY rare circumstanecs)- drives do it all the time; it's almost expected. I don't give a crap about another 20GB or $20 off, I want a hard drive that won't turn itself into a paperweight after a year or two. If I'm going to own the drive for 5 years, what's another $20?

SMART was an improvement, but most OS's(linux included) don't even recognize SMART info out of the box. Even if you've got the SMART utilities installed and the kernel modules etc, /var/log/messages is so noisy, I mostly ignore it- same for Win2k boxes, Event Manager is full of TONS of crap(thank god it has filtering, but still...) If SMART were to be useful, the HD would beep at you, or blink its LED, or the OS would annoy you with popup messages so you knew, "oh shit, I gotta back up my stuff to somewhere else, NOW!"

I had an ancient 4GB Digital drive I got second-hand, in the early 90's; it was already several years old when I got my hands on it, so it was probably pre-90's. It weighed a ton, took up the full space of a 3.5" drive bay, and even had its own little suspension system. I abused that thing to hell and back, carrying it in bookbags, cooking it when the fan on the external case died...the whole nine yards. I think I low-level formatted it a dozen times(something you're not supposed to do often on SCSI drives, supposedly). It only finally gave up the ship around '99, when it spent a couple months cooking itself to death hooked up "temporarily" to a machine I forgot about.

Meanwhile, I've lost two quantum drives(one laptop, one Ultra2 3.5") and my athlon's Maxtor drive is making funny noises every once in a while. None of them were more than 2, 3 years old TOPS. WTF? The excuse seems to be that consumers don't need the reliability corporate users 'demand'.

Home users users have, at the very least, equal needs as business users, because while businesses need to keep going 24x7, they often have backups, clusters, RAID units, etc. Most home users don't have any of their data backed up, RAID is practically unheard of among the jane-and-bob computer users, and of course no clustering.

minor detail...

[H0NGK0NGPH00EY]

I agree with your main point. I am a bit suspicious of your alleged 4GB drive from the "early 90's" though. I don't believe 4GB drives were even avaliable until the mid-90's. An interesting chart (that only applies to IBM drives) may be found here [storagereview.com].

Re:Processors = reliable, hard drives != reliable

[Wanker]

Here's some USENET perspective on hard drive history:

Jul 20 1990: 600MB about as big as you can find [google.com]

Apr 5 1992: 2gb disks mentioned as "new" [google.com]

Jul 13 1992: A mention of a Seagate 43400N (3.6GB) [google.com]

So if he means "before 1993" as early '90s, it could be valid. I doubt that it predates the 1990s-- certainly not in a 3.5" form factor.

Re:Processors = reliable, hard drives != reliable

[Dunark]

If SMART were to be useful, the HD would beep at you, or blink its LED, or the OS would annoy you with popup messages so you knew, "oh shit, I gotta back up my stuff to somewhere else, NOW!"

I don't think you'll be seeing that. My cynical opinion is that SMART is mostly a way to delay user awareness of a problem until the last possible moment - hopefully, after the warranty has expired.

Communication Speed

[blixel]

Too bad Communication Speeds for Networking and more importantly, Internetworking, haven't moved along any where near that speed. In my years of computing I've only seen the progression from 2400bps modems to 56k modems, and more recently to DSL and Cable.

Those speeds pale in comparison to the headway made in disk storage and CPU's.

That's nothing

[John Jorsett]

First computer I ever worked on that had a HDD was a Data General Nova 800. Drives then cost @$2000, and held 2.5 MEGA bytes on a thing the diameter of a super-duper extra-large pizza in a drive that required 4U space in a 19" rack. Given my life expectancy and the present rate of growth, I expect I'll see multi-exabyte drives. Amazing, but I nevertheless hope to be bitching about lack of capacity on my deathbed.

mass storage capacity doubles every 12 months

[RhettLivingston]

I pointed that out a while back. Furthermore, the industry has shown signs that they can push it to 9 when they want to.

I've also pointed out that the capacity will be easily used. First you'll want to record a full time video stream. Then multiples so that you can record the lives of your family and everything that occurs in multiple locations you own. The real hit comes when you start recording it in 3D and in enough resolution that you can later zoom on anything that was around you at any point in t

Latency is not really moving though

[Oestergaard]

Drives today have 10.000 rpm or 15.000 rpm. Eight years ago the high end was 7200 rpm, 5400 before that...

That's approximately a 2X performance increase per EIGHT YEARS. This is very very far from being impressive.

Disk seek time is dominated (today) by rotational latency. The fastest disks have seek times around 4ms, and that is pretty much the rotational latency on a 15000 rpm disk.

In order to improve disk performance (the seek time, not the throughput), disks need to spin faster. This does pose some interesting problems though...

A normal 3.5" drive has a platter with approximately 48mm radius, giving roughly 0.3 meter circumference. At 15000rpm the speed of the circumference is 75.4m/s.

Doing the math, this gives us a centripetal acceleration of v^2/r = 118435 m/s^2, or roughly 12085G. Sure as hell beats most drag racers out there (by more than a factor of 12000 ;)

The fun part is, that a simple doubling of the rotational speed, will do really interesting things to the acceleration (note the v^2 thing above).

A 30000rpm disk will have a centripetal acceleration of the circumference of approximately 48000G.

A mass-element at the circumference weighing one gram, will have a "pull" corresponding to (F=m*a) 118kg - which again will be approximately half a tonne on the 30000rpm disk.

You need to find a material that will weigh little, not deform under the given stress, and still have the necessary properties for use as a hard drive platter...

Re:Latency is not really moving though

[ottffssent]

You've got some problems with your facts. But, since you're playing the math games that I like to play, I'll cut you some slack. And then I'll expound.

First off, disk access time is dominated by actuator movement (seek time). Rotational latency on a 15,000rpm disk is 2ms, not 4. The fastest 15K drives have 3.5-4ms seek time. Slower drives have slower actuators, meaning the ratio of seek time to rotational latency is about the same, 2:1.

Seek time on large drives is of no importance. Seek time on small drives is of supreme importance. Small drives should be used to store the OS, applications, and small data files. Rapid access to disparate regions of the disk is important since these drives are primarily limited by IO/sec. Large drives are used for mass data storage. Large data storage (media, in my case) is dominated naturally enough by large files whereas applications and user data tend to be tiny. My media drive, for example has about 11,000 files in 95GB, or about 110 seeks/GB. My OS/apps drive, on the other hand, has over 89,000 files in 5.75GB, or 16000 seeks/GB.

Consider that a high-end drive can handle perhaps 600 IO/sec, and a large IDE drive can handle perhaps 150. Clearly then we have a problem: usage patterns differing by 150:1 in terms of number of seeks are not matched well to drives differing by 4:1 in seek performance. As you've demonstrated, physics cannot allow us to increase SCSI's seek performance to 150X that of bulk IDE drives.

The only way to achieve that sort of performance is with solid state storage. RAM costs about $150/GB - let's see someone mass-produce consumer-grade SSDs. Call it the "drive accelerator" and build it into a removable HDD bay. I guarantee that 1GB of RAM caching the most-used files on a hard drive would see performance skyrocket. Sure, it would be expensive, but it would be cheaper than the 15k SCSI boot disk I have, and a whole lot faster.

Re:Latency is not really moving though

[Oestergaard]

First off, disk access time is dominated by actuator movement (seek time). Rotational latency on a 15,000rpm disk is 2ms, not 4. The fastest 15K drives have 3.5-4ms seek time.

Example: Seagate 15krpm drive: average seek time 3.6ms. You are correct that the *average* rotational latency will be 2ms, since the full rotational latency is 4ms. However, 2ms out of 3.6ms is more than half, meaning rotational latency dominates (even though you were right about the average rotational latency being important, not t

Re:Latency is not really moving though

[ottffssent]

Example: Seagate 15krpm drive: average seek time 3.6ms. You are correct that the *average* rotational latency will be 2ms, since the full rotational latency is 4ms. However, 2ms out of 3.6ms is more than half, meaning rotational latency dominates (even though you were right about the average rotational latency being important, not the full rotational latency).

Seek time is 3.6ms. Access time is 5.6ms. The seek time is the time it takes for the heads to seek to the proper location. This is followed by (c

at this rate

[geekoid]

I'll be able to couple some hard drives to my flux capacitor and record the entire history of the universe.

Why is it no time traveller goes and says 'hi' to Jesus? Thats what I'd do.

The Math

[bigattichouse]

Moores is the # of transistors/processing power every 18 mos... you're looking at price per byte.

lets see what $100 gets you

$100/meg = 1985 10 meg
$50/meg = 1986.5 20 meg
$25/meg = 1988 40 meg
$12.5/meg = 1989.5 80 meg
$6.25/meg = 1991 160 meg
$3.13/meg = 1992.5 320 meg
$1.56/meg = 1994 640 meg
$0.78/meg = 1995.5 1.2 g
$0.39/meg = 1997 2.4 g
$0.19/meg = 1998.5 4.8 g
$0.09/meg = 2000 9.6 g
$0.04/meg = 2001.5 18.6g
$0.02/meg = 2003 37.2g
$0.01/meg = 2003.5 74.4g

Looks like the curve is a bit faster than every 18 mo... I think 12 months might be a better approximation of storage/cost.

for now

[SHEENmaster]

but wait until the RIAA starts charging in advance for piracy. They can do $15 for an album, or charge $15000 per song ammounting to $1,485,000 for a single recordable disc (99 possible tracks).

Re:for now

[Random Frequency]

I still say that levys against cds imply that I am a theif, and if I continue to pay the levys against cdrs, that I should be able to selectively download all the music I want, since I've paid for it anyways.

Oh well =)

Re:Wow!

[The Dobber]

Moores Law for Microchips
(doubles every 18 mnths)

Porns Law For Storage?

Re:yeah, but...

[Squarewav]

hard drives have gotten "faster and faster" as you put it, not nearly the same speed increes as cpus but still much faster my 60 gig dma133 can read a 40 meg file faster then then a 40 meg drive can even start seeking for the data, hell its at least 4 times faster then an old 8gig I have lying around. if your talking about the past few years I can agree with you as IDE caped out pritty much, unfortunetly the cost of moving to a faster media far out wieghs the "slowness factor" of IDE

Re:yeah, but...

[The_K4]

Um....there's SATA [serialata.org] on the radar screens :). It is worth the cost of moving to a faster media. :)

Actually, performance gains for disk drives ...

[Glonoinha]

Consider the IBM XT 4.77MHz with a factory default formatted 17 sector per track MFM hard drive with a 6:1 interleave. The peak throughput of this machine was roughly 87 kilobytes per second.

Now consider the new SATA machines with measured (not calculated) throughputs of 87 megabytes per second.

This is a 1,000x fold increase. For CPU processer throughput (speed) to keep up with this performance at the same rate, you would be able to buy a machine with a 4.77GHz CPU in it. Right now the fastest stock boxes are running what ... 3.06GHz, maybe a touch faster?

CPUs have gotten faster. Hard drives have gotten faster faster.

Re:Actually, performance gains for disk drives ...

[tchuladdiass]

Why is it that everybody continues to equate [M|G]Hz with CPU speed? It's only a component. Those old 4.77Mhz boxes took several Hz to complete a single instruction. A modern Intel or AMD chip runs several instructions per cycle. So therefore, a current top of the line system actually runs 3-5 thousand times faster than the original PC/XT. (But it still takes a couple to 3 minutes or so to boot up.)

Re:yeah, but...

[DuSTman31]

Well, they have in some ways.. Increases in capacity have come due mainly to vast increases in the areal density of the media. This, in turn, yielded massive increases in the rate the data moved under the heads..

The problem with hard disks isn't the data transfer rates they are capable of - it's their latency we need to worry about.

We need better defragmentation algorithms - I suspect that files are usually accessed in list order.. When running a program, for example, it's always going to want to read

Re:yeah, but...

[darkwhite]

This is achievable to a large extent by never installing programs on a non-defragmented disk. That way, a program writes all its files in a contiguous chunk, and assuming it will more or less want to access those files only, it has a degree of contiguousness.

Re:yeah, but...

[FireballFreddy]

Nope. A fragmented disk is what you get when you chuck your hard drive against a brick wall.

Re:yeah, but...

[dAzED1]

"microprocessors have gotten faster and faster. hard disks have not."

Good lord! Are you serious??? You obviously never had to use debug to partition an esdi drive. You obviously have somehow missed the whole transition from 10Mb burst to 160Mb burst we've seen in the last few years. Scsi has, as well, gone from 8-bit with 5Mb transfer (scsi1) to 10Mb transfer @ 8bit or 20Mb transfer @ 16bit (scsi2), 40Mb transfer @ 16bit wide (scsi3) and now 80Mb transfer @ 16bit wide with ultra2. And while that's just what the *bus* can handle, I can promise you that the disks of today are far faster than the disks of even just a year ago.

Ask yourself...what restricts data fransfer speed? Several things, really. Density is actually a factor, as its an engineerign feat to get the disks spinning fast, so the more bits go past the heads during a given time, the more can be put on/pulled off. Also, the ability to process that data, which - guess what, has significantly increased. Then there's the length of time a head needs to spend to actually get a bit to seat at a N/S, 0/1 - materials platters are made of are constantly being improved, so that's far better. Then theres the mamangement of the data itself, algorythms for where to write what, etc. Again, substantially improving, constantly. And all I've discussed was scsi - ide has improved (has quantity on its side) far more than scsi has the last few years, too.

How in the WORLD could you say hard disks haven't gotten faster? Oh wait, I know how...because you are either being sarcastic, you're insane, or you simply have no idea what you're talking about. Did you just start using computers last week?

Re:yeah, but...

[nomel]

I don't agree.

The physical drive passes by the head at a certain rate, depending on the speed of rotation of the platter and the distance the head is from the center of the platter.

Lets say 1 inch from the center, going 7200 rpm.
This means that the disk will be passing under the head at about

2 * pi * 7200 = inches per minute
/ 60 = inches per second
/ 12 = feet per second
((2 * pi * 7200) / 60) / 12 = about 62.8 feet per second 1 inch from the center of the platter.

Now, le

Hard Drives *HAVE* gotten faster...

[raehl]

Or at least, putting and retrieving data from hard drives has gotten faster....

Let's say you take that 12x of power over Moore's law, and instead buy 12 hard drives instead of one.

Now Raid-0 them.

Boom. Fast.

Re:Hum..

[CausticWindow]

vic acid is nice.. calx too.. blue calx, green calx, yellow calx :)