The Supercomputer Race 158
CWmike writes "Every June and November a new list of the world's fastest supercomputers is revealed. The latest Top 500 list marked the scaling of computing's Mount Everest — the petaflops barrier. IBM's 'Roadrunner' topped the list, burning up the bytes at 1.026 petaflops. A computer to die for if you are a supercomputer user for whom no machine ever seems fast enough? Maybe not, says Richard Loft, director of supercomputing research at the National Center for Atmospheric Research in Boulder, Colo. The Top 500 list is only useful in telling you the absolute upper bound of the capabilities of the computers ... It's not useful in terms of telling you their utility in real scientific calculations. The problem with the rankings: a decades-old benchmark called Linpack, which is Fortran code that measures the speed of processors on floating-point math operations. One possible fix: Invoking specialization. Loft says of petaflops, peak performance, benchmark results, positions on a list — 'it's a little shell game that everybody plays. ... All we care about is the number of years of climate we can simulate in one day of wall-clock computer time. That tells you what kinds of experiments you can do.' State-of-the-art systems today can simulate about five years per day of computer time, he says, but some climatologists yearn to simulate 100 years in a day."
Re:Flops not useful? (Score:5, Interesting)
That's just the problem, people want to hear raw numbers, but those are useless.
How well can it do the specific task it needs to do is the actual question. It's a hard one, to be sure.
Re:The true best measurement (Score:3, Interesting)
Re:Weather Day After Tomorrow (Score:5, Interesting)
44 years ago 1-5 megaflops was hot! What excitement we felt when the CDC6600 was installed at my university!
Back in '85 I was part of a startup building a mini-Cray, reimplementing the Cray instruction set in a smaller, cheaper box. I remember we focused on the Whetstone benchmark a lot, and it turned out that the Whetstone code really was bound up by moving characters around while formatting output strings, etc. We paid very careful attention to efficiently coding the C library string handling routines, and that got us more performance payback than anything we could do to optimize the arithmetic. One needs to understand the benchmark being used.
Like any benchmark... (Score:3, Interesting)
Just with a lot more dollars behind it...
Every one remotely engaged in Top500 systems knows how very specific the thing being measured is. It's most sensitive to the aggregate clock cycles and processor architecture, and not as sensitive to memory throughput/architecture or networking as many real world things are.
http://icl.cs.utk.edu/hpcc/ [utk.edu]
Is an attempt to be more comprehensive, at least, by specifying a whole suite of independently scored benchmarks to reflect the strengths and weaknesses of things in a more holistic way. Sure, it's still synthetic, but it can give a better 'at-a-glance' indicator of several generally important aspects of a supercomputer configuration.
The thing probably inhibiting acceptance of this is that very fact, that it is holistic and the winner 'depends' on how you sort the data. This is excellent for those wanting to more comprehensively understand their configurations standing in the scheme of things, but hard for vendors and facilities to use for marketing leverage. Being able to say 'we built *the* fastest supercomputer according to the list' is a lot stronger than 'depending on how you count, we could be considered number one. Vendors will aggressively pursue pricing knowing about the attached bragging rights, and facilities that receive research grant money similarly want the ability to make statements without disclaimers.
Rest assured, though, that more thorough evaluations are done and not every decision in the Top500 is just about that benchmark. For example, AMD platforms are doing more strongly than they would if only HPL score is counted. AMD's memory performance is still outrageously better than Intel and is good for many HPC applications, but Intel's current generation trounces AMD in HPL score. Of course, Intel did overwhelmingly become popular upon release of their 64-bit core architecture based systems, but still..
Is it *REALLY* the Top 500? (Score:2, Interesting)
Most of the locations listed are mostly educational institutions, r&d centers, and computer companies. The results were probably submitted unofficially. There are few exceptions, but they are just that--few. It makes you wonder what the Big Data companies (Google, Yahoo!, etc) actually have running. They have no reason to participate, after all...
Consider something like Yahoo!'s research cluster [yahoo.com]. Why isn't it on this list? Why don't they run the tests?
Well, let's see (Score:5, Interesting)
It's about a half a petaflop... but guess what? It runs Linux!
This sounds kind of nice but why should this make it any easier to write parallel programs for it? You still have to manage hundreds if not thousands of threads, right? This will not magically turn it into a computer for the masses, I guarantee you that. I have said it elswhere [blogspot.com] but parallel computing will not come of age until they do away with multithreading and the traditional CPU core [blogspot.com]. There is a way to build and program parallel computers that does not involve the use of threads or CPUs. This is the only way to solve the parallel programming crisis. Until then, supercomputing will continue to be a curiosity that us mainstream programmers and users can only dream about.
Re:Simulation (Score:3, Interesting)
Both these factors are hard/impossible to control and to standardize, and the only factor that does not change is the actual hardware and its peak/sustained performance, so it's the only sensible metric.
This is common knowledge (Score:3, Interesting)
It's fair to criticize Linpack for being a one-trick pony. It measures system performance for dense linear algebra, and nothing else. Jack Dongarra (the guy who wrote Linpack and maintains the top 500 lists) is quite up-front about Linpack's limitations, and he thinks that using a single number as the end-all-be-all of a computer's performance is a bad idea. It's a simple fact of life that certian kinds of computers do better on certain problems. The good guys out at Berkeley even sat down a couple years ago and enumerated [berkeley.edu] all of the problems they found in real-world HPC applications (See the tables on pages 11-12). The real truth here is that people should stop treating Linpack like it's the final measure of system performance. If you are doing pure linear algebra problems, it's a pretty good measurement for your purposes; if you are not, then you use it at your own peril.
Re:Flops not useful? (Score:3, Interesting)
"How can you possibly evaluate supercomputers in any other way except how many mathematical operations can be performed in some reference time? "
It's much more subtle than that. Most programs, including weather simulations, use a large amount of data stored on disk and in RAM. The problem with LINPACK as a benchmark is that, for all practical purposes, it ignores this cost by using a few very specific linear algebra operations that have very low communication/computation ratios. The LINPACK number is only relevant if your program is primarily based on operations that have this characteristic.
Unfortunately, most scientific codes (weather simulations included, of course), have evolved past simple implementations based on dense matrix-matrix multiplication (the particular kernel that gives the peak performance number) and include a number of steps that perform closer to the speed of the memory bus than the speed of the processor (sparse matrix operations, which make simulations tractable with millions of variables work this way). There's also the simple fact that very few programmers are even aware of the techniques required to achieve even 50% of peak performance on a kernel as simple as matrix-matrix multiplication. And, the cost of getting past 50% in programmer time is rather high. So, even if scientific codes could be optimally implemented, there's almost no chance they are.
Most people in HPC (myself included) have reached the point where the Top 500 list is a fun curiosity, but has little relevance to actual practice of supercomputing. Optimizing memory bandwidth and interconnects is much more important than raw FLOPS.
Still, I applaud the Roadrunner project. They took some serious risks to pull it off and created a very impressive computer. It's too bad that it will most likely be a one-off implementation (yeah, you can buy QS-22s from IBM, but I doubt they'll be around for too long).
-Chris
Please stop linking to Wikipedia (Score:1, Interesting)
We need a new moderation: -1 Wikipedia Googlebomb. Yes we know you can look things up in Wikipedia. But every time you make a link to Wikipedia from Slashdot, Wikipedia goes up in the Google Page Rankings. And then people act all surprised when Wikipedia is in the top ten for every Google search. Every time you link to Wikipedia, it gets a little bit more powerful.
So instead, why not link to some other relevant page [lanl.gov]? In this case, link to the owner of the Roadrunner supercomputer. You can probably even go to Wikipedia to get the link. If Wikipedia has a great page on something, don't link to it, just put the plaintext name in like this: "Search for IBM_Roadrunner on Wikipedia."
Please everybody, stop linking to Wikipedia. You're destroying the internet.
Re:I agree (Score:1, Interesting)
Yes, but Roadrunner uses the new Cell SPUs - very different hardware from a typical x86-based cluster.
Red Storm, if I recall correctly, runs a minimalist Linux kernel - something called Catamount, or Compute Node Linux, or something along those lines. Basically, it lacks some features of a full Linux kernel, including various I/O capabilities, but frankly I haven't heard of people having a hard time using Red Storm.
Isn't Modeling Weather Futile? (Score:2, Interesting)
I seem to recall a Nova special I watched many moons ago about "strange attractors" and "fractal behavior" that seemed to indicate that for a large class of complex-valued iterative functions there was a weird phenomenon called the "Butterfly Effect". Apparently... according to this show I saw 20 years ago (and I think that Mandelbrot mentioned it in a lecture I attended a few years later), initial variables which are as intertwined as the rational and irrational numbers can have drastically divergent outcomes in these situations.
It seems that the reason that this was called the Butterfly Effect was actually because the disturbance caused by a butterfly could be enough to change the track of a massive storm some days later. ( Reference [wikipedia.org])
The fact is that the weather forecasters on the local broadcast channel are less accurate than if they always predicted sun in one study:
"The graph above shows that stations get their precipitation predictions correct about 85 percent of the time one day out and decline to about 73 percent seven days out.
"On the surface, that would not seem too bad. But consider that if a meteorologist always predicted that it would never rain, they would be right 86.3 percent of the time. So if a viewer was looking for more certainty than just assuming it will not rain, a successful meteorologist would have to be better than 86.3 percent. Three of the forecasters were about 87 percent at one day out â" a hair over the threshold for success."
(ref: http://freakonomics.blogs.nytimes.com/2008/04/21/how-valid-are-tv-weather-forecasts/ [nytimes.com])
It's a wonderful idea that we can model the incredibly complex climate of our huge planet, but I'll believe it once I can trust the weekend forecast before Friday.
Any other ideas about useful purposes to put these huge computers to? Perhaps accounting and auditing for the new Emergency Financial Legislation?
Re:Financial modeling and spying better funded (Score:3, Interesting)
Sadly, while predicting the weather and better understanding it ultimately helps a lot of people, I suspect a LOT more computing power is thrown at more mundane things like predicting where the financial markets are going to be based on a gazillion data inputs. Probably even better funded are the vast datacenters around the world that fondle communications and other data for the spymasters. I doubt those computing resources are represented in the annual supercomputing lists. :)
There are a couple of misperceptions here.
Both the problems described, modeling years of weather models or modeling financial instruments, suffer from a definite flaw: they are not mathematical problems in the "high school" sense of the world, i.e. it is not possible to prove that there is only one finite solution that is demonstrably right.
Financial models are "fit to reality": you take a long time series, make a few wild guesses, throw it into a Cray-2, and look what the model says. Lather, rinse, repeat. they work, most of the times.....Bank regulators just recently allowed banks to discard simpler risk controls if they proved that they had a financial modeling tool that did not get their accepted measure of risk wrong more that 3 days out of 120, and these models were mostly sophisticated mathematicals and statistical implementations....and then Lehman goes belly up. [lehman.com] Mind you, they were using computer modeling as well.
[DISCLAIMER: I've been working in finance since 1988, and I believe equally in advanced financial modeling, the tooth fairy and Santa]
Weather modelling in yearly time scales suffers from the same flaw, in my view: unless you have a long enough set of possible inputs , it's not verifiable in reality.
If I try to make prediction over next week, I only need a week to see if my model is horribly wrong. If I want to test it in different seasons, 1-2 years can give me a good enough hint about the accuracy of a model.
If I am trying to predict average weather changes on a 20 years time scale, I need 20 years of historical data to get 1 (one) result. another year to get two, etc. exaggerating a lot, it's like modeling to predict red or black at the roulette: 50% of the models will get it right the first time, and 25% over two tries. definitely not enough to know if it works.