Cell-based "Roadrunner" Tops Elusive Petaflop Mark

prunedude writes "The NY times is reporting that an American military supercomputer, assembled from components originally designed for video game machines, is more than twice as fast as the previous fastest supercomputer, the I.B.M. BlueGene/L. To put the performance of the machine in perspective, Thomas P. D'Agostino, the administrator of the National Nuclear Security Administration, said that if all six billion people on earth used hand calculators and performed calculations 24 hours a day and seven days a week, it would take them 46 years to do what the Roadrunner can in one day."

Summary should have a shout out

[HolyCoitus (658601)]

1350 IBM Linux cluster team. xCAT for pwning.

Re:Summary should have a shout out

[DrEldarion (114072)]

From what I've heard (I'm no expert), these processors are good at certain types of calculations, but horrible at others. Ask a Cell to run Folding@Home, and it'll be blazing. Ask it to do general-purpose computing, though, and you'll quickly see the strength of other processors.

Re:Summary should have a shout out

[chrysrobyn (106763)]

From what I've heard (I'm no expert), these processors are good at certain types of calculations, but horrible at others. Ask a Cell to run Folding@Home, and it'll be blazing. Ask it to do general-purpose computing, though, and you'll quickly see the strength of other processors.

You're precisely correct. Cell's strength is in very predictable workloads (ones it can perform without branch mispredict penalties), very parallelizable workloads (ones that can be distributed over 6-8 SPU's / SPC's) that fit within 256 KB of local storage per SPU (manually managed cache, mapped to main memory). The non-double precision floating point enhanced version's (the version in the PS3) strength is further limited to integer and single precision floating point workloads. Roadrunner's Cell-DP eliminates that last limitation. While video games, encryption, nuke simulations and anything else that involves matrix manipulation can really stretch their legs on such a beast, general purpose computing won't find a benefit.

Re:Summary should have a shout out

[jacquesm (154384)]

The biggest problem is not to acquire the hardware (the graphics boards can be bought easily enough and some manufacturers make motherboards that will host 4 of them), the problem is in the software.

As you probably know your current desktop computer executes software 'in parallel' on multiple cores. Unless that software is written to use multiple cores then the parallelization will probably take place at the task level, in other words, unless special care was taken during the development each core will be executing a different process (or parts of a process, slicing itself up between multiple processes as long as the number of tasks is greater than the number of cpus).

The cell processor and the graphics cards you are talking about (I assume you refer to the nvidia 'cuda' architecture) are not your ordinary processor (though the cell comes a lot closer). They need special software to get the maximum out of their parallel features, which means that you can only exploit that speed when you are trying to solve a particular class of problems.

The issue is that not every problem is easily adapted to a parallel solution, and for some classes of problems it's simply impossible. For general purpose computing a general purpose cpu (with or without multiple cores) is good enough.

If you have a problem that you can solve in a parallel way (say computing chess moves, analyzing genetic strings, ray tracing an image or fluid dynamics, add your own examples here) then it makes sense to invest the extra time to code the problem up in such a way that if you have lots of cpus that they can each work on a small part of the problem.

The graphics cards are closer to a traditional vector processor than the cell, and thus even less suitable to be used as general purpose cpus.

Re:Cell processor

[BiggerIsBetter (682164)]

One thing to remember is that there is various iterations of the cell processor. The Xbox is a 3 core version The Playstation. I believe the Playstation is a 6 core processor. The roadrunner will use a 8 core processor. IBM originally discussed having a 16 core processor. There was not much talk about it afterwards. My guess is that there was significant bus contention issues. The original Power4's shut down one of their cores while running at full speed to avoid contention. The Power6 was designed to overcome these issues.

No, the XBox 360 has a three core PowerPC processor, not a Cell BE processor. The Cell BE in the PS3 has 1 PPE (Power core) and 7 SPEs (that "other" CPU core), while IBM apparently gets to use the fully functional Cells (PPE + 8 SPE) in their more expensive hardware. Those Cells with even fewer functioning SPEs might end up in HDTV TVs or similar.

Re:Cell processor

[bestinshow (985111)]

XBox360 has a tri-core in-order PowerPC - each core is actually very similar to the single general purpose PPU in the PS3's Cell.

Cell in addition has 8 SPUs. 1 is disabled in the PS3 for yield reasons, and another is reserved, so there are 6 available for general purpose computing.

Both run at 3.2GHz. I think Cell has at least 3x the vector/streaming power of the XBox 360 CPU, but only 1/3rd of the general purpose capability. Figures pulled from thin air, etc.

But can it run....

[CastrTroy (595695)]

By can it run Crysis?

Re:

[beav007 (746004)]

Can it even run Vista Ultimate?

Re:But can it run....

[nawcom (941663)]

Hah you think the military is dumb enough to even install Vista? Haven't you ever heard of military intelli... errm.. nevermind. Yeah, I'm sure they have it installed already.

Re:But can it run....

[Hojima (1228978)]

Windows: End Program - Untitled - Nuclear Defense. This program is not responding. To return to Windows and check the status of the program, click Cancel. If you choose to end the program immediately, you will loose any unprotected civilians. To end the program now, click End Now. Army personnel: Sir, I think we should send an error report.

Re:But can it run....

[pha7boy (1242512)]

military supercomputer, assembled from components originally designed for video game machines

yes. yes it can. and, most likely, it will also blend.

Re:But can it run....

[setagllib (753300)]

The marketing for that game was done all wrong. Insiders report that it was meant as an interactive preview of 3DMark2020.

exaflop, zettaflop, the yottaflop and the xeraflop

[anaesthetica (596507)]

Military taking the lead on computing as usual. Why is the military so much more progressive (with practical results) than any other institution of government?

It will be used principally to solve classified military problems to ensure that the nation's stockpile of nuclear weapons will continue to work correctly as they age. The Roadrunner will simulate the behavior of the weapons in the first fraction of a second during an explosion. Before it is placed in a classified environment, it will also be used to explore scientific problems like climate change.

So, it also has Cell-based processors AND Opterons. I wonder what the functional division between the two chip types is?

"If Chevy wins the Daytona 500, they try to convince you the Chevy Malibu you're driving will benefit from this," said Steve Wallach, a supercomputer designer who is chief scientist of Convey Computer, a start-up firm based in Richardson, Tex. Those who work with weapons might not have much to offer the video gamers of the world, he suggested.

Who cares? It's awesome sui generis.

Re:exaflop, zettaflop, the yottaflop and the xeraf

[Renraku (518261)]

The military is more progressive because there's not a whole lot they can do to advance things.

They can hope for random breakthroughs, mostly based on chance/luck/etc..

Or they can follow the natural progression of things. If you want to make things explode you have to know the nature of the explosion. And to know the nature of explosions you have to know all about high-energy physics at a molecular level. And to know about high-energy physics you have to know about how molecules and atoms interact. Now, with all of these things you can either make them yourself and study the real explosion, or you can simulate it and confirm with real-world results..which is what they're doing.

They have the resources AND the desire to do so, and therefore, they are doing so. Private industries will rarely do things like this on their own. They're much more likely to wait for someone else to do the research, or research with grants and then patent the results for their own profit. Its the same reason NASA has spurred many developments and improvements in the rest of the civilian world.

This setup will make it easier to study weather, physics, etc, etc. On the other hand, it'll also make it easier to figure out how to make bigger sticks that are lighter and sharper.

Re:exaflop, zettaflop, the yottaflop and the xeraf

[DogDude (805747)]

Military taking the lead on computing as usual. Why is the military so much more progressive (with practical results) than any other institution of government?

Are you kidding? [warresisters.org]

Re:exaflop, zettaflop, the yottaflop and the xeraf

[anaesthetica (596507)]

Are you kidding?

Not really. The post you link to describes the defense budget as it dwarfs other spending, but doesn't really argue why or why not that spending is progressive/regressive.

The military was one of the first racially integrated public institutions in the U.S., it researched and funded the Internet, it's pouring money into synthetic fuels right now, and it's pushing the limits of computing power as seen in this article. There are numerous other scientific and social areas in which the military advances society, with far more practical results than do-gooders in other government or public institutions.

Re:exaflop, zettaflop, the yottaflop and the xeraf

[blahplusplus (757119)]

"There are numerous other scientific and social areas in which the military advances society, with far more practical results than do-gooders in other government or public institutions."

It's because the military doesn't have the scrutiny and oversight other institutions do, lets face it. Do public institutions besides the miilitary get secret prison's and liscense to do whatever the want? The military is not held back by moral qualms. We've seen this with all sorts of classified documents coming out of the government. The military has budgets that are kept secret. For anyone to claim the 'military helps us' vs public institutions, we'd have to do an analysis. But that would be fairly difficult and politically sensitive, now wouldn't it?

Re:

[anaesthetica (596507)]

...secret prison's [sic]...not held back by moral qualms...

Are you really arguing that the scientific and social advances from the military arise from secret prisons and lack of moral qualms?

Re:exaflop, zettaflop, the yottaflop and the xeraf

[blahplusplus (757119)]

My point is they do not have the same barriers other institutions do: i.e. the gaps funding and scrutiny. My point about mentioning secret prisons was merely an example of the previous point.

Change in paradigm

[karvind (833059)]

If one looks at http://www.top500.org/ [top500.org] list and compare the CPU frequencies of the top supercomputers - all BlueGene CPUs were running at less than a GHz. And it seemed those low power cores were key to HPC (high performance computing). Cell and opteron - both run at multiple GHz and (presumably consume more power). IBM still has next generation of BlueGene/Q in works and is also for +Petaflop computation.

I feel bad for Whyle E. ...

[CaptainPatent (1087643)]

...There's no catapult in the world that will catch THAT roadrunner!

so what else is new?

[WheresMyDingo (659258)]

the roadrunner always wins, so it no surprise it topped this "petaflop mark" guy (yeesh, what a name).

and roadrunner's always been cel-based, at least in the modern era. i bought one of those cels from the warner bros. store before they went under, nice one too with his tongue sticking out

Perspective?

[Bob54321 (911744)]

To put the performance of the machine in perspective, Thomas P. D'Agostino, the administrator of the National Nuclear Security Administration, said that if all six billion people on earth used hand calculators and performed calculations 24 hours a day and seven days a week, it would take them 46 years to do what the Roadrunner can in one day."

That really put it in perspective for me. I normally judge a supercomputer by how many "all Earth people hand calculation years" it can do in a day...

Re:

[Digestromath (1190577)]

Normally I would compare computers by floating operations per second. However sicne I guess we are going back to the old style of comparing it to people doing calculations by hand. What about all the people on earth using abaci 24/7? Or by leagues per bushel over cubits squared?

ummm...

[WheresMyDingo (659258)]

if all six billion people on earth used hand calculators and performed calculations 24 hours a day and seven days a week, it would take them 46 years to do what the Roadrunner can in one day

probably because most of those people would either try to eat the calculator or sell it for food and medicine

Re:ummm...

[x2A (858210)]

Grr, we're already overpopulated without everyone on the planet multiplying t the same time.

Back it my day!

[by Anonymous Coward]

The kids these days are lazy, back in my day if we wanted to know if a nuke worked we'd take it out back test it!
Whatever happened to nuked marsh mellows or sitting round with Geiger counters trying to make funny sounds?
Kids are lazy these days!

Not in perspective

[justinlee37 (993373)]

To put the performance of the machine in perspective, Thomas P. D'Agostino, the administrator of the National Nuclear Security Administration, said that if all six billion people on earth used hand calculators and performed calculations 24 hours a day and seven days a week, it would take them 46 years to do what the Roadrunner can in one day.

That does not put the performance of the machine in perspective at all. Technical details would be much more accurate and effective.

Re:Not in perspective - this is a media number

[mykepredko (40154)]

This is a perfect example of a propellor head trying to come up with an analogy for a media/marketing type. I suspect that this was the only one that the powers that be felt non-techies could relate to. I've been asked to come up with these analogies a couple of times and it can be pretty frustrating on both sides.

I suspect the first example of this happening was trying to estimate how many angels could fit on the head of a pin.

Other meaningless analogies could be:

• How long it would take Roadrunner to count all the atoms in the universe
  
• What speed your car would run at if the speed difference between Roadrunner and your home computer was multiplied by 60 mph
  
• If we could go this many times faster than the speed of light, how fast could we cross the universe
  
• If in every instruction it could take in one byte of text, how long it would take to read the Library of Congress
  
• How fast it could render "The Incredibles" compared to how long it took the original server farm (actually, this might be one that's understandable)
  
• How fast it could break the 128 bit encryption used when you log onto your bank's web page to pay your bills (this might also be understandable and would probably be a bit scary)
  

The simple fact is that a petaflop computer works faster than humans can conceive and any kind of analogy cannot be comprehended.

myke

Re:Not in perspective - this is a media number

[jareds (100340)]

How fast it could break the 128 bit encryption used when you log onto your bank's web page to pay your bills (this might also be understandable and would probably be a bit scary)

No, not at all scary. It's apparently twice is fast as the BlueGene/L, which apparently set a record of 478.2 teraFLOPS. Let's assume it takes 1 floating-point operation to test a single key, which is a gross underestimate. We'll thus assume the Roadrunner can test 10^15 keys per second. Testing 2^128 keys would then take about 10^16 years.

Take that petaflop with a grain of salt

[jimhill (7277)]

As a software developer who's worked on the Lab's previous ASC machines (Blue Mountain, Q, Lightning) I can say that once the calculation is run to get a machine atop Jack Dongarra's gee-golly list, it's partitioned, segmented, divided, and subjected to such crappy resource management that if I could trade the entire machine for a pair of coupled 8-core Mac Pros I'd do it in a heartbeat.

The real PITA with these machines is that the powers that be are trying to kill two birds with one stone: they want an R&D platform for advanced computing, but they also want to certify an aging and untestable nuclear stockpile. That rather requires a fairly static platform, and so far our experience with ASC has been that when a machine hits that sweet state, they yank it and give us the next one.

The result of their research:

[suck_burners_rice (1258684)]

The answer is 42. The question is left as an exercise for the reader.

WTF happened to plain old units of measure?

[PseudoThink (576121)]

if all six billion people on earth used hand calculators and performed calculations 24 hours a day and seven days a week, it would take them 46 years to do what the Roadrunner can in one day

I'm glad to see the continuing trend of creatively "dumbing down" units of measure (in this case, flops) to the point where they are not only practically useless, but entirely divorced from reality. I would like to propose the following similar, hype-worthy measure for fuel economy:

Old: Miles per gallon
New: Number of miles from which one would smell the excrement from the number of cattle one could feed for a day with the amount of corn it would take to produce one gallon.

Yes, It Does Run Linux

[Doc Ruby (173196)]

As I posted the last time this story was reported (in IBM Touts Supercomputers for Enterprise [slashdot.org]") in "Yes, It Does Run Linux" [slashdot.org]:

From IBM's detailed press release [ibm.com]:

the QS22 boasts an open environment, utilizing the flexibility of Red Hat Enterprise Linux as the primary operating system and the open development environment of Eclipse.

That means that a PS3 running Linux [psubuntu.com], even with its ridiculously low 512MB RAM, can be used as a $500 development platform for these CellBE BladeServers.

And, in turn, some QS22 SW might be usable on the PS3, if it can be ported to use the tiny RAM. Or if someone hooks an i-RAM bank to the SATA port as swap/ramdisk, using perhaps iSCSI over its Gb-e for storage.

Now get out there and supercompute!

NOT MILITARY!

[serviscope_minor (664417)]

It's the department of energy, not the military. Specifically, it is at Los Alamos, which is not a military base.

Re:

[francium de neobie (590783)]

They're trying to pull 1000 times your lab's results.

Re:Question

[avalys (221114)]

It will be used for nuclear weapons simulations - primarily for investigating issues related to how warheads will perform as they age.

Re:Question

[Anthony Rosequist (1110043)]

Before it does weapons simulations, it will first work on some scientific problems, like model testing to predict climate change.

After it's done with that (I wonder how they will determine what done is...), it will go classified and do nuke simulations.

Re:

[by Anonymous Coward]

And I've heard that republicans eat babies. As someone who has worked with climate models including data collection I can safely say you're full of shit. There are thousands of research stations collecting the data. For it to be generally corrupted, there'd have to be some vast global conspiracy whereby publically competing research stations and countries agree to privately skew their data.

Now there IS something of a vast global conspiracy (PNAC, Republicans, Bilderberg, etc), but, er, it's not on the p

Re:Question

[Brian Gordon (987471)]

Quick, make it play tic tac toe against itself.

Re:Question

[kylehase (982334)]

Perhaps they should invest in a computer to track warhead parts.

Re:Question

[x2A (858210)]

The previous model used hundreds of dual core P4s, just running NOP's at full speed. The heat generated, being equivelant to that outputted by a nuke, meant they could run simulations without having to actually write any code.

Re:Question

[attemptedgoalie (634133)]

Lets say you have designed a nuclear weapon.

Wouldn't it be really neat to run some tests before you build it?

For instance, how cool would it be to have a simulation that could test a weapon being mishandled, or shot. At every single point from every possible angle at every possible velocity?

It would be nice to know that there is a possibility of detonation if it were to drop off of a loading rack.

Re:Question

[anaesthetica (596507)]

What exactly would the military use a supercomputer for?

The military will use this advanced technology to assist and perhaps automate the RTFA process, also known as Reading The Fucking Article, which would allow you to answer your query without posting.

Re:Question

[ScrewMaster (602015)]

the only thing I can really think of is the air force doing the obvious shady things that it does.

Uh ... what exactly do you mean by "shady things"? If you have a problem with what our armed forces are doing, you'd be better off leveling your charges at Congress. Ultimately, they're the ones that fund any "shady" things the military does.

calculators

[hansoloaf (668609)]

I bet if everyone had the TI-57, it'll take the aforementioned 46 years.
But the TI-68 will cut it down to 23 years.

Re:

[by Anonymous Coward]

It was designed originally for the PS3. But not solely for it.

Cell was the brainchild of Ken Kutagari of Sony and Peter Hofstee of IBM.

Re:The future

[Thalin (130318)]

This is actually based on Cell 2 or as IBM marketing likes to say it "Cell eXtreme"!

Cell 1 (the Playstation chip) didn't have the double precision floating performance to achieve the petaflop mark; Cell 2 is far better on that front.

This CELL is not single precision

[raftpeople (844215)]

The CELL processor is single precision, which translates into wrong answers most of the time. Me guess is that for problems requireing double-precision numerics, you should divide CELL based supercomputer by 10 to 100 (software emulation of double precision is MASSIVELY SLOW), so this is really a teraflop machine. No big whoop...NEXT!!!

Things move fast in technology Jethro, including this 2nd gen of the CELL proc, this is what you missed:

Double Precision FP - 190TFLOPS (5 times faster than 1st CELL)
Memory: Expanded to 32gb
Memory: DDR2 instead of Rambus
65nm (I know, I know, but it's better than 90nm)

Allow me to oblige ...

[Zero__Kelvin (151819)]

Take everyone on earth, and put them each in a different Ferrari Testarossa [wikipedia.org] with no engine, no gas in the tank, and no ignition system. That is how fast this thing moves.

Some other equally useful analogies:

Take the same aforementioned people, and give them a OLPC. The amount of time it takes them all to calculate their degree of separation from Kevin Bacon [wikipedia.org], and divide by a googolplex [wikipedia.org]. , then round up. That is the number of people that think the calculator analogy in the article was a good one.

Take the inverse of the clock frequency and multiply it by the number of instructions required for Windows to boot far enough to attempt to obtain an IP Address dynamically. Add to that the time it takes for the DHCP request to reach your Billion made router [apcmag.com]. That is the amount of time it takes for it to hose your router. Take the inverse of the clock frequency and multiply it by the number of instructions it takes to apply a service pack. Add it to the boot time, calculated as described above. That is the amount of time it takes to achieve a BSOD.

HTH,

- Thomas P. D'Agostino