NNSA Supercomputer Breaks Computing Record

Lecutis writes "National Nuclear Security Administration (NNSA) Administrator Linton F. Brooks announced that on March 23, 2005, a supercomputer developed through the Advanced Simulation and Computing program for NNSAs Stockpile Stewardship efforts has performed 135.3 trillion floating point operations per second (teraFLOP/s) on the industry standard LINPACK benchmark, making it the fastest supercomputer in the world."

Neat

[neccoant (3345)]

It's amazing that we were stalled at 50TFLOPS for two years, and are piling on the FLOPS now.

Re:Neat

[the_2nd_coming (444906)]

the increased flops is simply a function of the fact that they are expanding the number of nodes.

Re:Neat

[brsmith4 (567390)]

That's not how linpack works. Sure, increasing your number of nodes will give definite performance advantages to course-grained, embarassingly parallel applications, but Linpack is not one of these applications. As well, Linpack should not be used as a guide for raw floating point performance, but is much better suited to gauge throughput.

Linpack does its benchmarks using a more fine-grained algorithm, creating lots of communications for Message Passing to share segments of dense matrices for rather large linear systems. Not only is the number of nodes a factor, but so is the interconnect speed. If that cluster was using GigE for its interconnect, its Linpack benchmarks would not be nearly as impressive. Haven't RTFA but its likely that BlueGene/L is using Myranet or Infinband for its interconnect (or possibly a more proprietary backplane style interconnect, though that cluster is way too big for that).

These latest generations of high-speed interconnects (esp. Infinband) have brought clusters closer to the point of being near shared-memory performance and hence is more of a throughput test than anything else.

This description of the HPL benchmark (The "official" name for the Linpack benchmark) should provide some clarity as to how memory-dependent Linpack actually is:

The algorithm used by HPL can be summarized by the following keywords: Two-dimensional block-cyclic data distribution - Right-looking variant of the LU factorization with row partial pivoting featuring multiple look-ahead depths - Recursive panel factorization with pivot search and column broadcast combined - Various virtual panel broadcast topologies - bandwidth reducing swap-broadcast algorithm - backward substitution with look-ahead of depth 1.

http://www.netlib.org/benchmark/hpl/ [netlib.org]

They took a lot of time to get Linpack to be less shared-memory dependent, like adding the swap-broadcast algorithm (which i'm fairly certain was absent in the old mainframe version of Linpack), to make it more "fair" to run on a cluster versus a shared memory set up. However, on a typical cluster, Linpack can push your interconnect pretty hard, esp. if you are stuck on GigE. However, Linpack has _lots_ of settings and parameters to "tune" the benchmark for your particular cluster.

My point: Linpack/HPL is not an overall flops benchmark for a cluster. It measures the performance not only of double precision CPU performance, but also the performance of a cluster's interconnect.

Re:Neat

[imsabbel (611519)]

Well, in fact the truth is a right in the middle.
Linpack is VERY easy to parallize. Earth simulator and other vector machines get over 85% of their theoretical processing power with linpack, and even clusters with relatively abyssmal interconnects are still in the 50% range.

Lots of computational problems need orders of magnitutes more inter-node communication, up to the point where linpack doesnt even matter anymore and clusters and vector computers with the same linpack score are a factor of 10 or 20 apart.

Re:Neat

[brsmith4 (567390)]

Were you correcting my spelling? Because I always make that mistake (myranet... it's myrinet damn it!). You know what I meant though ;) It looks like BlueGene/L is using a hybrid backplane/hypertorus interconnect where a whole bunch of "machines" (more like system-on-a-chip) are connected via a backplane, then that case of "machines" is connected to another case in the same rack on some number of layers of interconnect. Then the racks are connected using some other protocol. Though you may not "be at l

Re:Neat

[by Anonymous Coward]

I think they were waiting for final specs on Doom3.

Re:Neat

[woah (781250)]

The reason is, of course, that we've been stuck with sameish desktop performance as well. Which correlates with supercomputer performance, since nowdays most of them use Intel/AMD processors.

Just goes to show that Moore's law won't hold forever.

Re:Neat

[JQuick (411434)]

Actually Intel compatible clusters in the supercomputer rankings are not all that compelling. True, linux cluster did fare very well for several years as measured by price/performance. Also it is true that about 63% of the top 500 supercomputers are Intel or Intel compatible.

Despite this, the majority of systems at the top of supercomputer top 500 chart are based on the POWER architecture, not Intel chips.

The POWER based systems, including BlueGene and PowerPC systems, are all much better on both price/p

Re:Neat

[imsabbel (611519)]

You are misstaken.
We didnt STALL at 30Gflops, its just that the 30Gflops were SO much better than everything else available that it took a couple of years to catch up and overtake it.

If you average over the last 10 years, the the Earth simulator was a bump above moores law and now we are back on track.

and its only half the machine too!

[rebelcool (247749)]

wait till its fully online.

Re:and its only half the machine too!

[RicktheBrick (588466)]

It would seem to me that with the great accuracy and concrete penetrating weapons that we possess that we could establish a future date when the United States would destroy all of our nuclear weapons. I believe that we could do this unilateraly. I believe we could wipe out all the leadership of any country that dared to launch an attack on our country and hopefully we will be able to intecept their missile before they reach our cities. I do not believe that we can morally tell anyone not to possess nuclear weapons unless we dedicate ourself to a date when we do not possess them too.

Re:and its only half the machine too!

[Henry V .009 (518000)]

I do not believe that we can morally tell anyone not to possess nuclear weapons unless we dedicate ourself to a date when we do not possess them too.

Then the solution is to immorally tell other nations that they can't have them. That way you have neither the stupidity of unilateral disarmament nor the stupidity of looking the other way on nuclear proliferation. I hope that this bit of Life 101 helps you out there.

Re:and its only half the machine too!

[Profane MuthaFucka (574406)]

Easy cowboy, before you start proclaiming that we could wipe out all the leadership of an arbitrary country, let's just get Osama first. OK?

Re:and its only half the machine too!

[einhverfr (238914)]

You have a point. Nuclear weapons were a heavily stabilizing force in the cold war because they made it impossible for any leader to consider all-out war with the other country. THings are more complex now, and nuclear proliferation is a different issue. Yet it is not a simple issue. On one hand, nuclear weapons continue to help prevent horrible war crimes like the firebombings of civilian population centers (Dresden, Tokyo) because it is simply too risky to do this. Yet they themselves are effective s

Re:and its only half the machine too!

[InadequateCamel (515839)]

I get your point, but I think that to describe this in terms of police is a little inaccurate because that really is the police's job. Perhaps this is closer to a heavily-armed civilian militia (I hesitate to say "Mafia" or "gang", but you could argue points there too) taking on the police role?

From the press release...

[Zebra_X (13249)]

This performance was achieved at Lawrence Livermore National Laboratory (LLNL) at only the half-system point of the IBM BlueGene/L installation. Last November, just one-quarter of BlueGene/L topped the TOP500 List of the world's top supercomputers.

Is there anything that will be able to touch this when it's complete?

Re:From the press release...

[Daniel Boisvert (143499)]

The closest I've heard of is the Cray X1E, but even that only claims [cray.com] 147 TFLOPS.

Re:From the press release...

[As Seen On TV (857673)]

The X1E isn't intended to be a fastest-in-the-world supercomputer. It's intended to be a low-cost scalable vector system. The fact that it's fast is great, but it's not its main design feature.

Now, the X2, on the other hand, is a whale. They're talking 150 TFLOPS at roll-out next year (unimpressive) and 300 TFLOPS after the block 10 update the year after that (very impressive).

Of course, the X2 isn't working yet, so who the hell knows. But it's fun to think about.

Blue Gene?

[eth8686 (793978)]

Didn't IBM push Blue Gene to 180'something teraflops recently?? News story herer [businessweek.com]

Re:Blue Gene?

[EBorisch (530762)]

This is Blue Gene. Read the article...

This *is* Blue gene.

[daveschroeder (516195)]

RTFA [doe.gov]

Or, at least the article's title:

"NNSA Supercomputer Breaks Computing Record: Exceeds 100 TERAFLOPS DOE/NNSA and IBM partnership on BlueGene/L, a tool for national security"

imagine

[dario_moreno (263767)]

a Beowulf cluster of these !

Wow!

[FlyByPC (841016)]

Just imagine running Fractint on this puppy!

Re:Wow!

[ucblockhead (63650)]

Heh. I guess I wasn't the only one who christianed a new machine by running fractint on it. Gave it up around 1998 because there was just no point.

Steroids

[tiktok (147569)]

There was another machine that had already beaten that record, but unfortunately failed a diagnostic test for banned substances...

Did you RTFA?

[Donny Smith (567043)]

> has performed 135.3 trillion floating point operations per second (teraFLOP/s) on the industry standard LINPACK benchmark, making it the fastest supercomputer in the world."

Did you read the fucking article?

"This performance was achieved at Lawrence Livermore National Laboratory (LLNL) at only the half-system point of the IBM BlueGene/L installation. Last November, just one-quarter of BlueGene/L topped the TOP500 List of the world's top supercomputers."

See, this is the SAME supercomputer that has already topped the list last November, so the latest record did NOT make it the fastest supercomputer in the world.

It already had been the fastest supercomputer in the world.

Yeah, but...

[Black Jack Hyde (2374)]

...will it run NetHack [nethack.org]?

Earth Simulator

[by Anonymous Coward]

I rather miss the time when the world's most
powerful supercomputer was used to study our
planet. It was something to be proud of, actually.
These machines are essentially weapons. Pity, that.

Re:Earth Simulator

[lp-habu (734825)]

Historically, I'll think you'll find that a great many technological advances were made with the original purpose of killing other beings -- usually other humans. Seems to be one of the basic human characteristics. Pretty effective, too.

OK then...

[caveat (26803)]

How 'bout we use Blue Gene for climate modeling, and start setting off full-yield nuclear tests to insure the viability of the stockpile? I don't terribly like the idea of nukes, but the genie is out of the bottle and there's no stuffing it back in - we need to have the things, and if god forbid we ever have to use them, I'd like to see them work properly. Seriously...unless you use one of the interconnect cables to garrote somebody, these computers are hardly "weapons", quite the opposite in fact.

Link to the list

[dnaboy (569188)]

FYI the top 500 supercomputers list is maintained at http://www.top500.org/ [top500.org].

Dupe

[karvind (833059)]

Didn't we cover this before [slashdot.org] ?

More Computer Power=Fewer Nuclear Explosions

[Nova Express (100383)]

To study the effects of different nuclear weapon designs, there are basically two approaches:

1. Throw massive amounts of computing power at the problem (as done here), or:
2. Actually set off a nuclear weapon.

Having massive computing power in the hands of Lawrence Livermore scientists reduces or even eliminates the need for U.S. nuclear forces to actually detonate nuclear and thermonuclear explosions.

Of course, some people would prefer to see the United States undertake unilateral nuclear disarmament, something they've been advocating since SANE/FREEZE was telling us we could trust the Soviet Union in the 1980s. Only today they claim we can trust Kim Il Jong and the mullahs of Iran more than the democratically elected government of the United States, just as they claimed we could trust Leonid Breshnev and Yuri Andropov more than we could trust Ronald Reagan. Their views are every bit as ill-conceived now as they were then.

Re:More Computer Power=Fewer Nuclear Explosions

[ozborn (161426)]

Of course, some people would prefer to see the United States undertake unilateral nuclear disarmament, something they've been advocating since SANE/FREEZE was telling us we could trust the Soviet Union in the 1980s. Only today they claim we can trust Kim Il Jong and the mullahs of Iran more than the democratically elected government of the United States, just as they claimed we could trust Leonid Breshnev and Yuri Andropov more than we could trust Ronald Reagan. Their views are every bit as ill-conceived now as they were then.
Nice strawman you've constructed, but pray tell who are these "some people" you are talking about? I challenge you to cite a single press release, webpage or publication by any independent NGO (even kooky ones) pushing for nuclear disarmanment that claims Kong Il Jong can be trusted. I can't think of any disarmament/peace group that would be opposed to 3rd party bilateral weapons inspections.

DOE's Senior Activity Center

[Animats (122034)]

The "stockpile stewardship program" is basically a senior activity center for retired physicists. They have busywork projects to keep people thinking about how to design nuclear weapons. DOE is worried that all the old bomb designers will die off, and no new ones will replace them.

Remember, everything in the inventory was designed with far less compute power than today's desktops.

Re:DOE's Senior Activity Center

[T5 (308759)]

DOE's stewardship program is not for retired scientists, but current ones. The laboratory directors at the nuclear labs (Sandia/LLNL/maybe others) are required to certify the stockpile as being ready to go each year. Their supercomputers are the only way to test the aging stockpile without actually detonating a few to see which designs age better than others.

And let's remember that almost everything in the current arsenal was designed and actually tested, not just worked up via computer. It takes a whol

Can we qualify this a bit?

[Moderation abuser (184013)]

"making it the fastest supercomputer in the world"

Or rather the fastest supercomputer with published LINPACK results. There are a number of reasons that agencies with supercomputers might not want to publish results.

135.3 trillion floating point operations per seco

[alex4u2nv (869827)]

135.3 trillion floating point operations per second

Does this mean we can't slashdot it?

You of course know what this means.

[Allnighterking (74212)]

You can now open a Mozilla session in under a minute!

The belly of the beast

[theufo (575732)]

Here's an article describing some of the specs.

http://www.llnl.gov/asci/platforms/bluegene/talks/ gupta.pdf [llnl.gov]

It's from the days when BlueGene/L was still relatively small, but the basic design hasn't changed since then.

Turns out it's split into I/O and computing nodes. The 1024 I/O nodes run Linux. Each controls 64 dual-cpu nodes, which use simplistic microkernels written from scratch using Linux as an example.

The network architecture sounds funky: apparantly it's based on a torus!

You Gotta Love This...

[Nom du Keyboard (633989)]

Scientists at LLNL for the first time have performed 16-million-atom molecular dynamics simulations with the highest accuracy inter-atomic potentials necessary to resolve the key physical effects to successfully model pressure induced rapid resolidification in Tantalum.

You just gotta love a sentence like that!

Re:hmmmmm...

[a1cypher (619776)]

Just for a point of reference, does anybody know how many floating point operations a 3.2ghz processor can do per seccond?

I know its not 3.2billion because most micro operations take at least 3 or 4 clock cycles.

Re:hmmmmm...

[Yartrebo (690383)]

With SSE instructions, you can process 4 floats at once, so I'm guessing that 3.2 GHz processor can do a few gigaflops.

Re:hmmmmm...

[tgamblin (873085)]

Depends on the problem and the memory performance as much as it does on the GPU. There's no good answer to that question. For kicks though, this paper [stanford.edu] has some measurements for matrix multiply using ATLAS. It's comparing a Pentium 4 to an NV40 GPU. The P4 wins at about 7 GFlops, and the NV40 loses due to horrible memory performance. That's pretty ironic considering that the NV40 has quite a few more FPU's, and that they're in parallel. It's a good example of why you can't ever say for sure how a processor's going to perform until you test it on a real workload.

Re:hmmmmm...

[fafalone (633739)]

A 3.2GHz Intel Xeon processor performs 6.4gflops, but clock speed isn't the only determining factor.

You are not seeing the larger picture

[ravenspear (756059)]

This is the same flawed logic that people try to apply to NASA on here a lot, that we shouldn't send people to the Moon/Mars until we fix all the problems on earth.

Halting scientific research to worry about all of our other problems is the wrong approach for many reasons. It is often scientific advances which lead to improved quality of life in many other areas of society.

Re:More important issues

[tgamblin (873085)]

Despite the fact that BlueGene/L is being built to simulate nukes, this kind of research does impact some of these other issues, and there is government money going into them. Here are some examples... The National Center for Atmospheric Research uses supercomputers [ucar.edu] to simulate effects of pollution and global warming, and projects like LEAD [ou.edu] are using grids with supercomputers attached to predict weather. Check out some of the projects [renci.org] at RENCI, as well. There's NIH-sponsored genetic research in addition to the weather stuff.

It may be sad that we live in a world where nuclear weapons research is driving the computing power, but it doesn't mean that the power of BlueGene/L isn't going to be used for thousands of other peaceful scientific applications, too.

Re:Human Intelligence?

[Kethinov (636034)]

Isn't the human brain supposed to be equivalent to a supercomputer running at about ~100 teraflops? And if so, shouldn't this computer be smarter than us?

In Star Trek TNG 2x09 [halo43.com] Data was quoted at having a total memory capacity of somewhere around 90 petabytes with a total linear computational speed of 60 trillian operations per second.

One would say this supercomputer is already more than twice as smart as Data!

Re:AMazing

[mikael (484)]

Or we can model protein folding even faster.
Run more accurate climate simulations even faster.
Run population simulations even faster.
Run CAD/CAM simulations even faster.

Re:AMazing

[brsmith4 (567390)]

A slightly larger dose of logic would tell you that NASA has nothing to do with this cluster, that it belongs to the NNSA or the National Nuclear Security Agency. They are probably more interested in testing new reactor designes or running simulations to demonstrate the effects of an aircraft crashing into one of their reactor domes (though I honestly believe that no one really believes that will happen).