Stories
Slash Boxes
Comments
Search

News for nerds, stuff that matters

Slashdot Log In

Log In

Create Account  |  Retrieve Password

Cray Introduces Adaptive Supercomputing

Posted by ScuttleMonkey on Fri Mar 24, 2006 02:02 PM
from the adapting-to-complexity dept.
Supercomputing Hardware
David Greene writes "HPCWire has a story about Cray's newly-introduced vision of Adaptive Supercomputing. The new system will combine multiple processor architectures to broaden applicability of HPC systems and reduce the complexity of HPC application development. Cray CTO Steve Scott says, 'The Cray motto is: adapt the system to the application - not the application to the system.'"
+ -
story
This discussion has been archived. No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.

Cray Introduces Adaptive Supercomputing 50 Comments More /

 Full
 Abbreviated
 Hidden
More
Loading... please wait.

  • Good Motto (Score:5, Insightful)

    by ackthpt (218170) * on Friday March 24 2006, @02:04PM (#14989792) Homepage Journal

    Cray CTO Steve Scott says, 'The Cray motto is: adapt the system to the application - not the application to the system.'

    That's a good motto, but how often do you bend the will of your application, needs or business to the limitations of the application? I've been sitting on something for a couple weeks after telling someone "You really should have accepted the information the other way, because this new way you want it is highly problematic (meaning: rather than rip it off with a simple SQL query, I'll have to do an app)"

    IMHO adapting to the needs of the user == customisationg, which also == money. Maybe it's not a bad idea at that! :-)

    In certain cases, at run-time, the system will determine the most appropriate processor for running a piece of code, and direct the execution accordingly.

    This assumes, of course, that you have X number of processors to chose from. If you can't do it, the answer is still 'throw more money at it, buy more hardware.'

    my head is still spinning from all the new buzzwords overheard at SD West 2006.

    • They just left out the ending. It's really: 'The Cray motto is: adapt the system to the application - not the application to the system.'" Why? Because hardare costs more to change!"

    • Co-processors anyone? (Score:4, Insightful)

      by TubeSteak (669689) on Friday March 24 2006, @02:26PM (#14989979) Journal
      After exhaustive analysis Cray Inc. concluded that, although multi-core commodity processors will deliver some improvement, exploiting parallelism through a variety of processor technologies using scalar, vector, multithreading and hardware accelerators (e.g., FPGAs or ClearSpeed co-processors) creates the greatest opportunity for application acceleration.
      So they're saying that instead of faster/more generalized processors, they want several specialized processors.

      Old ideas are new again.

      • Re:Co-processors anyone? (Score:5, Interesting)

        by sketerpot (454020) <sketerpotNO@SPAMgmail.com> on Friday March 24 2006, @03:20PM (#14990396)
        There are actually processors out there with compilers which can compile a few bottleneck C/C++ functions into hardware on an integrated FPGA. This expands the CPU instruction set in application-specific ways and can, in some cases, give absolutely enormous speedups.

        In other words, they're working on processors which are programmed in general-purpose languages, but which adapt their hardware to the specific program.

    • This assumes, of course, that you have X number of processors to chose from. If you can't do it, the answer is still 'throw more money at it, buy more hardware.'

      Except that in this case, one of the options they are promoting is FPGA - Field Programmable Gate Array - which can literally be adapted to the problem by reprogramming them. They are, in effect, an infinite number of processors to choose from.

      Look to see FPGA's showing up as coprocessors on more prosaic AMD Opteron motherboards in the near future
      • Are FPGAs fast now? That was always the problem before: you could create a custom processor, but it wouldn't run faster than 15MHz. If your FPGA can't keep up with the data stream, then what's the point? You'd probably be better off trying to hand-roll some custom microcode for one of your commodity processors. It's not as flexible as a PGA, but it'll be faster, and may be more appropriate for certain classes of problems.
        • Look on Xilinx's website. The Vertex4's (although currently having supply problems) go up to 500MHz (though you probably don't want to run anything at that speed considering that's probably the reg-to-reg limit). These things are literally better system-on-a-chip solutions than any ASICs could be considering what it offers. Integrated micro-processor, bus architecture, peripheral interfaces and non-volatile and volatile memory, with enough pins (BGA package) to expand with off-chip components. Actel eve
        • There are FPGAs over 250MHz now. There are times when such a beastie is useful. There are times when they aren't. Not sure why the hell you'd want to put this in a commodity PC though. It couldn't possibly help more than a second processor, which would be cheaper - or a second core, which would be cheaper still.
      • FPGA's aren't the golden answer currently. Most if not all FPGA's have issues with being used this way. They are programmable, but they're not made or intended to be programmed in the field (despite their name). The majority have a programmable life of maybe 1000 flashes with flash-based FPGA's (ProASIC from Actel for instance) having a life of maybe 100 flashes. They're basically a poor-man's ASIC more than anything else. The technology would have to improve significantly in a much different direction
    • This assumes, of course, that you have X number of processors to chose from. If you can't do it, the answer is still 'throw more money at it, buy more hardware.'

      Uh, sure. That would be the assumption. Just like back in the day you had math coprocessor for floating point operations, and now it's on the chip; or disk compression duties used to be handled by a Stacker board. Or today you have a graphics accelerator to handle 3D video.

      Sounds like a sound business too. Instead of having completely different

    • Re:Good Motto (Score:5, Interesting)

      by dildo (250211) on Friday March 24 2006, @03:19PM (#14990394)
      It is possible to build comptuers that are optimized for certain kinds of calculations.

      For example, Gerald Sussman of MIT (a computer scientist) and a Jack Wisdom (a physicist) decided they wanted to do long-term modelling of the solar system's evolution over time. Long time modelling of a multi-body system requires a fantastic amount of calculation. What is the best way to do it?

      Sussman and Wisdom came up with a crafty idea: build a computer that is specially configured at the hardware level to do the modelling. Sussman and his colleagues decided that with off-the-shelf parts they could build a computer that would be just as or more capable of modeling this system than a supercomputer would be. The result was the Digital Orrery, a relativlely cheap computer that gave great results. (It is now featured in the Smithsonian museum.)

      Think of it: if your computer is going to be doing the Fast Fourier Transform 6.02x10^23 times per day, why not build a superfast chip that does nothing but the FFT rather than express it as software? It's a pretty cool idea. I think this is the sort of thing that Cray computers claims to want to do with its motto.
    • 'The Cray motto is: adapt the system to the application - not the application to the system.'

      Wasn't it essentially that motto that once gave us the CISC architecture, which most people today agree was't such a great idea...?

      • Re:Adaptive = Adapting for Survive (Score:4, Informative)

        by some damn guy (564195) on Friday March 24 2006, @11:42PM (#14992609)
        Cray already makes systems based on many thousands of opteron processors. You can't beat them for scalar processing power. But what they also make,and still excel at, is specialized vector machines that can work with them. It's two good, but different tools for different jobs. The improvement is to make the two even more integrated and more flexible.

  • Cray? (Score:3, Funny)

    by Eightyford (893696) on Friday March 24 2006, @02:10PM (#14989847) Homepage
    I didn't even know Cray still existed. Maybe it was Sony's "emotion engine" [wikipedia.org] that almost killed them. ;)

  • Coolest Looking Supercomputers (Score:5, Interesting)

    by Eightyford (893696) on Friday March 24 2006, @02:13PM (#14989868) Homepage
    Cray always made the coolest looking supercomputers. Here's an interesting bit of trivia:
    The Cray T3D MC cabinet had an Apple Macintosh PowerBook laptop built into its front. Its only purpose was to display animated Cray Research and T3D logos on its color LCD screen.
  • "All of these platforms will use AMD Opterons for their scalar processor base.'

    Im just loving the vendors picking up on AMD.

    Their idea seems very interesting in theory. It sounds like HPC's version of the math co-processor->crypto accelerator idea.

    And at least they are not basing the userland on Unicos :)

  • Complexity, current machines (Score:4, Interesting)

    by gordyf (23004) on Friday March 24 2006, @02:20PM (#14989939)
    It seems like the idea of combining multiple architectures into a single machine is already being done -- we have fast general purpose CPUs (single and dual core x86 offerings from AMD and Intel), paired with very fast streaming vector chips on video cards, which can be used for other non-graphical operations like a coprocessor.

    The only difference I see is that they're relying on an intelligent compiler to decide which bits to send to which processing unit, but I'm not sure how much faith can be placed there. Cray certainly has a lot of supercomputing experience, but relying on compiler improvements to make or break an architecture doesn't have a good track record. I'm curious to see how they fare.

    • The only difference I see is that they're relying on an intelligent compiler to decide which bits to send to which processing unit, but I'm not sure how much faith can be placed there.

      If you read further into the article, you would have noticed TFA talks about a new programming language called "Chapel".

      Chapel was designed as a language for rapid development of new codes. It supports abstractions for data, task parallelism, arrays (sparse, hierarchical, etc.), graphs, hash tables and so on.

      So, they aren't re

    • Cray certainly has a lot of supercomputing experience, but relying on compiler improvements to make or break an architecture doesn't have a good track record.

      Seriously. Just ask Transmeta. Or Intel (think Itanic).

    • Re:Complexity, current machines (Score:4, Interesting)

      by flaming-opus (8186) on Friday March 24 2006, @03:16PM (#14990372)
      They really aren't rellying on compiler improvements, so much as passing the code through their vectorizing compiler, and a tool for generating their fpga codes. If the code optimization for these 2 steps fails to optimize very much, you bail out and send it to the general purpose (opteron) processors.

      Your being fairly pedantic about the computer architecture anyway. Yes, pairing multipe processor types together is not new, but most mpp supercomputers use identical node types.

      The jist of this story is simpler than it sounds. Cray has 4 product lines with 4 cpu types, 4 interconnect routers, 4 cabinet types, and 4 operating systems. They would like to condense this down. The first step is to reuse components from one machine to the next. There are distinct advantages for keeping the 4 cpu types for various problem sets, but most everything else could be multi-purpose. From the sounds of things, it's using the next generation of the seastar router in all of the machines. Thus you use the same router chips, cabling, backplane, and frame for all the products. This reduces the number of unique components cray has to worry about. If they go to DDR2 memory on the X1 and mta, that further simplifies things, though I suspect they won't.

      Well, once you share parts, why not make a frame with a bunch of general purpose CPUs for unoptimized codes, and a few fpga or vector cpus for the highly optimized codes? It allows customers more flexibility, and introduces cray's mid-range customers to the possibility of using the really high-end vector processors currently reserved for the high-end X1 systems. It's also a win for the current high-end customers. On the current X1 systems, you have these very elaborate processors running the user's optimized application, but the vector cpu's also end up running scalar codes like utilities and the operating system. These are tasks the vector cpu's aren't terribly good at, and you're using a $40,000 processor to run tasks a $1000 opteron will do better. Even if the customer isn't interested in mix-n-match codes on the system, (which I'm skeptical any cray customer really is), you probably want to throw a few dozen opteron nodes into the X1's successor, just to handle the OS, filesystems, networking, and the batch scheduler.
      • I wonder whether the Cascade vector processor will really be a stand-alone vector processor or actually a co-processor?

        Paradoxically, I believe that the major problem in making a really good vector processor is in pushing the envelope in single thread performance as well (Amdahls law and all that). In comparison, parallelism is easy. So in that sense, it would make sense for Cray to rely on AMD and x86 market volume to get good single thread performance very cheaply, and then concentrate resources on making
        • The X1 processor is already a coprocessor. Not in the sense that it's on a different piece of silicon from the scalar unit, but that the vector CPU's instruction stream is distinct from the scalar unit. In past cray systems, some cpu's used the same functional units for the scalar unit and vector unit, (T90) while some (J90) used distinct scalar units. The X1 is a vector unit bolted on the side of a MIPS scalar core, with synchronization logic, and multi-ported register files to support multi-streaming. I d

  • This begs the question... (Score:5, Funny)

    by creimer (824291) on Friday March 24 2006, @02:27PM (#14989992) Homepage
    So if I want to run Mine Sweeper, Cray will adapt one of their supercomputers to the requirements of this game? Sweet!
  • LJ had a good article on this a few months back.

    http://www.linuxjournal.com/node/8368/print [linuxjournal.com]
  • Cray as a company in general is amazing, they have been around forever and they dont sell bulk crap computers, go figure... The inspiration behind Cray is definitly worth a good study for future computer industry companies.

  • building machines around problems (Score:4, Interesting)

    by deadline (14171) on Friday March 24 2006, @02:42PM (#14990121) Homepage
    Cray finally figured it out. I have been saying for years:

    HPC/Beowulf clusters are about building machines around problems

    That is why Clusters are such a powerful paradigm. If your problem needs more processors/memory/bandwidth/data access, you can design a cluster to fit your problem and only buy what your need. In the past you had to buy a large supercomputer with lots of engineering you did not need. Designing clusters is an art, but the payoff is very good price-to-performance. I even wrote an article on Cluster Urban Legends [clustermonkey.net] the explains many of these issues.

    • That's not really true either.
      You only buy as much hardware as you need, but hardware is only half the cost of a computer like that. Infrastructure (physical, hardware, administrative, and management), software, and planning is a big part of the cost. Every time you install a capability-class machine, you plan for it several years in advance, make space/power/cooling available for it. Hire people to manage the machine. Port your applications to the machine. Install the machine, test and benchmark. Then admi

  • And What If... (Score:4, Insightful)

    by Nom du Keyboard (633989) on Friday March 24 2006, @02:46PM (#14990146)
    The new system will combine multiple processor architectures

    And what if I don't want multiple processor architectures, but instead just lots and lots of the single architecture my code is compiled for?

    • Then buy one of the older designs built on a single architecture?
      This announcement is telling us that they are planning to make new designs available which is not the same as saying that the old designs are no longer available.

    • Re:And What If... (Score:4, Insightful)

      by flaming-opus (8186) on Friday March 24 2006, @03:24PM (#14990428)

      The idea is that all the CPU types will be blades that all use the same router, and plug into a common backplane, and that the cabinets all cable together the same way. In all cases, I imagine there will be opterons around the periphery, as I/O nodes and running the operating system. Then you plug in compute nodes in the middle, where the computer nodes can be a bunch more opterons, or vector cpu's, or fpga's, or multithreaded cpus. There will certaintly be plenty of customers only interested in lotsa opterons on cray's fast interconnect, and they just won't buy any of the custom cpus.
    • Because, if you did, you'd realize that this environment has it's own tool chain and isn't for pre-compiled binaries.
    • Then you only buy one processor type (presumably the Opteron). As for precompiled binaries, assuming they meet the appropriate requirements (yet to be determined because predicting what API's will be necessary in a few years is pointless), they'll run just fine. However, optimizing for your target platform is likely to get you better performance regardless of whether the target comes from Cray or anyone else.

  • Buzz word. (Score:3, Insightful)

    by mtenhagen (450608) on Friday March 24 2006, @03:50PM (#14990624) Homepage
    While I must admit "Adaptive Supercomputing" sound like a realy cool buzz word, in practice the programmer still will need to adapt the application to the physical distribution of the systems. Or are they going to dynamicly rewire the switches?

    There have been several attempts (hpfortran, orca, etc..) to automate parallisme but most of them failed because a skilled programmer could create a much faster application within a few days. And remeber that a 10% performance boost in these applications means thousands of dollars saved.

    So I suspect this is just a buzz word.
      • 2 years behind in announcements, let's see who brings it to market first.

        Sadly the answer is that it's not even a race. SGI brought foreward their first step already, but won't get past that. You can now buy an fpga blade to put in your altix. While cray is just now announcing a unified vision for this, they've already had their fpga based solution since they bought octiga bay 2 years ago.

        As much as cray is suffering financially, SGI is in much worse shape, and they have about $350million in debt around the

    • Unless 'computing power' is different to 'combined processor speed', I don't understand what Cray are up to here..

      Well yes, they are very different. Processor speed is clock rate and tells you precisely jack shit about how much work can actually be done. Computing power is better measured in operations per second. Typically we measure integer and floating point performance separately. Even those benchmark numbers are usually pretty useless; hence we have the SPECint and SPECfp benchmarks which suppose

        • Depends on how parallelizable the problem is. Clusters work well on jobs that are highly parallelizable, and in which there's small inputs for large outputs. Supercomputers are better for jobs with large data sets that are not easily split up into smaller pieces that can be sent to cluster nodes. The major advantage is that all the processors have direct access to main memory instead of having to fetch data across the network.
        • No, you're not right.

          You're ignoring one *major* element of scientific computing vs. something like SETI@home. SETI requires *very* little communication over the network and *no* communication between the peers doing the computation. The kind of scientific computation that someone would do on a high end cluster or supercomputer (say, a weather model or a model of an airplane) requires a great deal of communication and, even more important, synchronization. Think about what happens if you have 10,000 proc

        • "ok, I shouldn't have said anything about processor speed. am I right in thinking that there's still no reason this adaptive approach will neccessarily be any better than a distributed project?"

          REally 2 different tools for 2 different problems. Distributed computing is really only useful for highly parallel tasks that require a LOT of computation on very little raw data. Furthermore, it has to be on data you don't mind shipping off to joe anonymous to be processed.

          Current and future crays are typically run
    • I don't understand what Cray are up to here.. perhaps someone can enlighten me?

      Imagine a beowulf cluster of Cray supercomputers!

    • That seems like a good idea, but you end up with a "one trick pony" that does only one thing really well.

      From my understanding most supercomputers are built to do one task... Either it be fold proteins, simulate nuclear explosions, predict weather, simulate the big bang, or various other number crunching task. By the time they are done, they have to move to the next project... Mostly because of being out of date every 5 years.
    • i think a more precise interpretation would be:

      (iteratively) adapt the system to the application (mix)

All trademarks and copyrights on this page are owned by their respective owners. Comments are owned by the Poster. The Rest © 1997-2009 Geeknet, Inc.