Slashdot is powered by your submissions, so send in your scoop

 


Forgot your password?
Close
typodupeerror
×
Programming IT Technology

Grid Computing: Conceptual Flyover For Developers 79

Posted by Hemos from the relating-to-the-known dept.
An anonymous reader writes "This article relates many Grid computing concepts to known quantities for developers, such as object-oriented programming, XML, and Web services. The author offers a reading list of white papers, articles, and books where you can find out more about Grid computing."
This discussion has been archived. No new comments can be posted.

Grid Computing: Conceptual Flyover For Developers More

Grid Computing: Conceptual Flyover For Developers

Comments Filter:

  • So what exactly is "grid computing"? (Score:5, Insightful)

    by evil_one666 ( 664331 ) on Monday November 01, 2004 @08:09AM (#10684652)
    So what is exactly "Grid Computing"? From the article...
    Sometimes it's easier to start defining Grid computing by telling you what it isn't. For instance, it's not artificial intelligence, and it's not some kind of advanced networking technology. It's also not some kind of science-fictional panacea to cure all of our technology ailments.

    If you can think of the Internet as a network of communication, then Grid computing is a network of computation: tools and protocols for coordinated resource sharing and problem solving among pooled assets. These pooled assets are known as virtual organizations. They can be distributed across the globe; they're heterogeneous (some PCs, some servers, maybe mainframes and supercomputers); somewhat autonomous (a Grid can potentially access resources in different organizations); and temporary.


    Nope- that still does not tell me what "grid computing" is. This vague, loosely defined definition can describe just about every "next big thing" since the mainframe.

    • Re:So what exactly is "grid computing"? (Score:1, Insightful)

      by Anonymous Coward
      Grid computing is utilizing multiple independent machines of different processing power in order to complete computational tasks.

      yeah its vague but that narrows it sopmewhat.

    • Re:So what exactly is "grid computing"? (Score:4, Informative)

      by carnivore302 ( 708545 ) on Monday November 01, 2004 @08:17AM (#10684687) Journal
      Hmm. It's not that hard, but I agree: the author seems to obfuscate a fairly simple to explain principle.

      In a few words: grid computing is the use of many connected computers for one task.

      Or, you might want to think about it as multithreading, but spread out over multiple machines.

      The author is making a case for a standardization of how this should be handled.

      • Re: So what exactly is "grid computing"? (Score:5, Informative)

        by Alwin Henseler ( 640539 ) on Monday November 01, 2004 @08:50AM (#10684857)
        "In a few words: grid computing is the use of many connected computers for one task. Or, you might want to think about it as multithreading, but spread out over multiple machines."

        In a way it's a matter of taste, but I'd define it this way: "parallel" -> many CPU's, but quite close in 1 place, like in a SMP desktop. "distributed" -> with network in between, as in Beowulf cluster (possibly over the internet).

        What would make "distributed" a Grid? The fact that it's 'everywhere', always working/available somewhere, like P2P networks. You can take your equipment off the network, but the network (ehh, grid) goes happily on doing its thing.

        This becomes really useful when it's a easy to use and commonplace as the internet today. Send out some software, it grabs a piece of data here, grabs a program there, finds a server to do the computation, and reports back to you with the result. Got some cycles to spare? Put some in the Grid, earn money. Just wait and see, some day computing power will be supplied and consumed the way electric power is today.

        • Just wait and see, some day computing power will be supplied and consumed the way electric power is today.

          No it won't. Computing power is still going to increase exponentially and reduce in price as per Moore's statement. With the advent of new material science, such as diamond microprocessors, the power of each individual processor will continue to improve.

          The added flexibility of x86ish based clustering will allow for small businesses to increase their ability to get work done and outcompete the la
          • Agree with you partially. Let me take an example: for electric grids, I've heard a figure once of 4% for average losses due to power distribution. That means, central production of electricity doesn't profit from economy of scale right away: it would have to be at least 4% more efficient to make it worthwile, and any advantage is decreased with 4% due to the distribution losses. Al things being equal, local production would be 4% more efficient, because it avoids the distribution losses.

            With networked co

            • With networked computing, there is a similar burden, and I suspect this burden is a lot more significant in the case of computing power. Network bandwidth is always a relatively limited, expensive resource, compared to computing power that you can install locally (and Moore's law is still going strong).

              Two things:

              1) Moore's law won't keep going forever.

              2) Networking is currently going faster than Moore's law.

              It is true that the relative amounts of CPU and networking available at any given moment do mat
        • I believe that the difference between "Grid" and "Distributed" computing is more subtle, but very easy to explain.

          If you need to have ONE memory image, with all CPU's using the exact same memory image at the exact same time, then that's Grid computing.

          If you have many CPU's, each with their own memory image, and they can perform their work no matter what the other CPU's are doing, then that's Distributed computing.

          Put this way, I believe you can really see the differences between Grid and Distributed

      • This isn't quite right. Grid computing is:

        1) coordinates resources that are not subject to centralized control & (A Grid integrates and coordinates resources and users that live within different control domains for example, the user s desktop vs. central computing; different administrative units of the same company; or different companies; and addresses the issues of security, policy, payment, membership, and so forth that arise in these settings. Otherwise, we are dealing with a local management sy

    • Re:So what exactly is "grid computing"? (Score:5, Interesting)

      by lachlan76 ( 770870 ) on Monday November 01, 2004 @08:20AM (#10684700)
      Keep going:

      Consider this: most IT departments are being forced to do more with less. Budgets are tight, resources are thin, and skilled human resources can be scarce or expensive. To top it off, most corporate managers know that they have a super-abundance of idle computing power. It's well known in industry circles that most desktop machines only use 5% to 10% of their capacity, and most servers barely peak out at 20%. No surprise then that many of the big money people in corporate America balk at the thought of purchasing more equipment to get the job done. What these companies need is not more horsepower, but more efficient use of existing horsepower. They need a way to tie all of these idle machines together into a pool of potential labor, manage those resources, and provide secure and reliable access to the number-crunching muscle. Imagine if a corporation or organization could use all of its idle desktop PCs at night to run memory- and processor-intensive tasks? They would get more work done faster, possibly get to market faster, and at the same time cut down their IT expenses.

      The idea seems to be to turn the whole network into a cluster. "Why buy more servers when you can gove some of the load to your desktops?" is a short summary.

      • The idea seems to be to turn the whole network into a cluster.

        This is answered in the FAQ at gridcomputing.com [gridcomputing.com]:

        The key distinction between clusters and grids mainly lies in the way resources are managed. In case of clusters, the resource allocation is performed by a centralised resource manager and all nodes cooperatively work together as a single unified resource. In case of Grids, each node has its own resource manager and don't aim for providing a single system view.

    • Grid computing is the concept of using distributed resources as one big resource. For example, Boise State currently uses all of the computers in the Engineering labs as a super computer when classes are not in session. Micron Technology uses all of its desktop systems as one big super computer.

      Todd Tannebaum just gave an exec lent keynote at Boise State's HPC Workshop. He explained that while computing power has increased on a system by system basis, the total available computing power to a single person
    • Grid Computing - simple.

      As far as the user is concerned, you have 1 Black Box system containing everything.

      The physical implementation could be a single supercomputer, or a whole host of different systems spread all over teh place. But the the end user, it's just a single computer that handles all their stuff for them.

      That's a grid. Check out http://wwws.sun.com/software/n1gridsystem/ for a good overview of how this can be implemented.

      Remember, a true grid system is more than just raw CPUs, it's all a

    • Re:So what exactly is "grid computing"? (Score:2, Informative)

      by Anonymous Coward
      Think of grids as a distributed set of clusters. Instead of using distributed networks of single PCs you connect clusters.
      The aim is get rid of the usual limitation to clusters (homogenious hardware/os) and make use of all the clusters in a virtual organization (think university network, distributed.net, seti@home protein folding and similar research institutions). You can then use a resource broker to specifiy you resource, say all big-endian machines with more than 1 gb of memory or all pentium machines.

    • Nope- that still does not tell me what "grid computing" is. This vague, loosely defined definition can describe just about every "next big thing" since the mainframe.

      I like to think of "grid computing" as the idea of building technology with the global use in mind: pluggable security models, standard protocols for job management/data transfer/etc., and so on. how to build services with the future (where the future is sharing those services) in mind.

      grid computing MAY take the form of: shared supercompu

    • Ian Foster's definition is here http://www-fp.mcs.anl.gov/~foster/Articles/WhatIsT heGrid.pdf [anl.gov]

      A book he edited is often considered the an excellent starting point, The Grid: Blueprint for a New Computing Infrastructure. His website http://www-fp.mcs.anl.gov/~foster/ [anl.gov]

      Only reason why I know anything is because I have a 10 page report due before the end of the week for my High Performance Computing module. Yippee.

    • How about "geographically distributed, heterogeneous parallel processing".
    • Its certainly a buzzword and have lots of potential. But currently its too immature to make any sense. Its Some times avoiding certain techonologies like CORBA and RMI and making its own protocols. But it seems like an amalgamation of Distributed cmputing and Parallel computing with newer protocols(like RFT,GridFTP, SOAP) and architecture (SOA). Its certianly supposed to be more productive than the systems its imitating but at this point its too fragile.

  • Change can be hard (Score:5, Interesting)

    by millahtime ( 710421 ) on Monday November 01, 2004 @08:10AM (#10684655) Homepage Journal
    Recently I saw a similar design for a network and some "old timers" said it was no good to do it this way. It wouldn't satisfy the needs.

    One thing I have noticed is that for many "old timers" there is the feeling of we have always done it the old way, why change. Any thoughts of how we drag that old donkey into the new methods when they don't want to go?

    • Re:Change can be hard (Score:5, Insightful)

      by Timesprout ( 579035 ) on Monday November 01, 2004 @08:18AM (#10684688)
      Well this is the thing. Many old timers are not adverse to change when clear benefits can be shown from adopting the new technology.

      Unfortunately for Grid Computing its still in the stage where people are struggling to explain what it is, nevermind what it does or how it can improve life. Thats always going to be a hard sell to me anyway. If its function is not obvious it makes you sceptical just how necessary it is.
      • It's going to be painful and resisted until it "just works" (TM), and then everyone will want to use it. So the trailblazers will most likely be techies who become suits, and want to prove themselves by saving a business loads of money by doing this. Probably banks, non?

        The overheads will be enormous though, at least initially. Security issues, data issues, even logging issues have to be thought through. Any system that implements this will be *fucking* complicated to work through.

        In any case, the "old

      • It's easy when classically trained/experienced to see tasks as captive procedurally to one processor. Numerous processors, distributed computing, and as a consequence, grids of computers, see a daemon, or a number of distributed but controlled daemon, as capable of broadening the number of concurrent tasks that can be performed. This is the benefit of the grid; resources that can be allocated and run concurrently, then added to the performance of the originating idea/program.

        It's a distributed hierachy of
      • I laughed when IBM said they were spending a billion dollars on grid computing, and I laugh now. Grid computing as something you can sell isn't going to be successful. People have really powerful computers these days, and buying lots of them is cheap. Put that together with peer-to-peer and what's the common excuse for paying someone money to use their computers? And could the providers actually make enough money for it to be profitable?

        Perhaps if they can add on some value to the basic service, there

  • Why is slashdot pushing grid computing so much? (Score:4, Informative)

    by Tedium Unleased ( 764661 ) on Monday November 01, 2004 @08:12AM (#10684662)
    The linked article is written in May 2003 yet it's new now?

  • Unready Hype (Score:5, Interesting)

    by Anonymous Coward on Monday November 01, 2004 @08:12AM (#10684664)
    As part of a university group that adopted Grid computing about a year ago, the Grid is mostly over-hyped material that isn't ready for prime time. The basic idea (see e.g. Legion) worked more than a decade ago, but what I've seen of today's Grid software is fragile, overcoupled, underdocumented, and doesn't yet deliver on all the promises.

    We were taught that the test of research software is whether a full professor (or corporate executive or other obscenely busy person worth >> $100/hour) finds it useful enough that they take time to learn it - the uses I've seen for the Grid don't pass that threshold yet.

    There are some exceptions: tightly-integrated applications put together in a couple of the hard sciences that really just do supercomputing with a friendlier face. There's enough payoff there for a physicist to be happy with the software.

    For a geek, however, even there, most "grid UI research" is simplistic, derivative, and uninspired.

    Apologies to my first-ever-advisor who is now a Grid bigwig. :)
    • but what I've seen of today's Grid software is fragile, overcoupled, underdocumented, and doesn't yet deliver on all the promises.

      Doesn't sound all that different than a lot of software I know.

    • Re:Unready Hype (Score:2, Informative)

      by Anonymous Coward
      As former Director of Operations for a large Grid facility, I agree with you, and I think most of my colleagues would as well.

      It has been standard practice over the past couple of years to overhype Grid, a practice which I suppose was intended to bootstrap interest but which instead just tends to leave people feeling confused and vaguely betrayed as they discover that what was presented as a production capability turns out to be a research project in its early stages. The article is typical of the approac

  • Grid (Score:2, Funny)

    by cuteseal ( 794590 )
    "What is the matrix?"
    "No Neo, try again"
    "What is grid computing?"
    "Bingo."

    There's certainly alot of info to devour there, but I guess if companies like Google and Dreamworks are using it, then it has to be a Good Thing.

    • I dont know about google, but I believe the dreamworks rendering used for shrek2 (that was theirs, right), was deployed onto a supercluster of 500+ nodes, not some fancy grid fabric.

  • Security in Grids (Score:5, Interesting)

    by ifoxtrot ( 529292 ) on Monday November 01, 2004 @08:15AM (#10684674)
    I have had some experience with grids and the overwhelming difficulties I've come across have been in the areas of security.

    First and foremost, grids are designed to run in a distributed environment which makes security design and administration that much more complex.

    Second, grids are currently in their infancy and there is little prior art to the types of attacks and problems that will affect them. Despite this, they are very juicy targets with the kind of storage and bandwidth that would make even a hard-bitten cracker weep for joy. (i apologise for the imagery)

    Third, in my book security has to be a top-down approach - i.e. the guys on top lead the way and then everyone else follows. Grids have no tops or bottoms which makes this a bit tough to apply. In short there is no security hierarchy in a default grid environment. Responsibility HAS to be established explicitly. A simple example is who is responsible for the data held on one of the nodes? Is it the person who wrote the application, the person who owns the application, the person who owns the hardware?

    Grids are fascinating in their security requirements (and those who think these are solved by web services have another thing coming! People are a huge aspect of the security of a system, and distributed system like grids have a very complex task of ensuring that people behave the way they should).

    • First and foremost, grids are designed to run in a distributed environment which makes security design and administration that much more complex.

      I'd have to say that in its essence, grid security is just traditional security on a larger scale with heavy X.509 use. sure, there's federation ideas and such, but there's nothing nothing "new" and unique to grid computing. we already have tons of users using public key authentication (agent systems, like "proxies" in grid-speak) from a distributed set of sit

  • Ancient Technology (Score:1, Insightful)

    by Anonymous Coward
    Is this news? Article is dated May 2003.

  • Grid Computing is a buzzword (Score:5, Insightful)

    by joib ( 70841 ) on Monday November 01, 2004 @08:20AM (#10684701)
    Like, if you fit in "grid computing" in your grant proposal, the probability that you'll get funding increases. Now, if in addition to "grid" you manage to fit in "nanotechnology", "bio-informatics" and "paradigm" you'll be funded with a probability very close to 100 %!
    • Note that this also trumps "web services" in Top Trumps Buzzword, since it includes it...
    • Do you BreatheEatAndSleep GridComputing? Does your Nanotechnology experience RockYourX Paradigm? Passionate about BioInformatics? Read on!

      SpreadThin is a BayAreaStartup with a Mission. The Mission: to combine Nanotech, BioInformatics, and GridComputing to create the NewParadigm for MoleculeBasedServices! Your Task: to synthesize these concepts into a Marketable NewParadigm. RoomAndBoard + Equity.

      Knowledge of WikiEmergencyMaintenance is DesirableButNotRequired. StartingImmediately.

      Resumes in HR-XML Resume

  • Well it's ONE view on Grid Computing (Score:4, Informative)

    by joelethan ( 782993 ) on Monday November 01, 2004 @08:33AM (#10684768) Journal
    This is meant to be a primer, and it just about "primes" the debate on Grid Computing.

    The grid discussed here seems only to be built on the OGSA and Globus Toolkit, and Globus has not really covered itself in glory with their poor UIs etc.

    Grid seems to address occasional demand for "much more power" from your computing resource, but does not really provide a consistent flexible computing resource.

    The academic world uses External Grids to pool resources [bioinformatics.org] but private Enterprise has little to gain from these External Grids in exchange for a HUGE security problem.

    And Internal Grids? These are so immature as to beggar belief. Why risk investing in these configurations when bang per buck is so uninviting.

    /joelethan

  • Stuff like this whereby you get a load of co-operating computers and a multi level archtitecture to utilise it was done years ago , all built on top of RPC. This to me is just a nother refashioning of age old ideas so the people involved can justify their research positions and so IBM (and others) can make a whole heap of cash out of gullible IT managers.
    • Yes and No.

      I think the difference is the attempt to scale this thing up no end; all the data coming of the CERN LHC will be grid processed; this is one of those problems where the grid is ideal: The data is not secret, you need lots of CPU power and you have PhD students to throw at the engineering of the whole thing.

      Classic web sites and the like are a different problem, one where the goal is not so much infinite scalability, but infinite flexibility. The deployment stuff I work on is designed to deal [smartfrog.org]

    • Stuff like this whereby you get a load of co-operating computers and a multi level archtitecture to utilise it was done years ago , all built on top of RPC

      Yes, and that's why it didn't work: thinking about distributed computing as a bunch of procedure calls that happen to be remote is wrong. The sad thing is that a sizeable number of people still thinks it's the way to go (e.g., all the SOAP adherents).

      This to me is just a nother refashioning of age old ideas so the people involved can justify their re

  • Practical application... (Score:3, Interesting)

    by BobRooney ( 602821 ) on Monday November 01, 2004 @09:43AM (#10685319) Homepage
    As an "boots on the ground" IT professional it would be nice to have a consumer grade "grid computing" solution to offer some small business customers as an alternative to buying a server farm for the two days a month they actually put strain on it.

    If there were an easy way to cluster their workstations they wouldnt need to invest in an underutilized server farm. They could just schedule their processor/disk intensive reports and processes for off hours or rely on grid load balancing to take the extra cycles from the computer of the CSO (Chief Solitaire Officer) so that the impact would be imperceptible to the average user.

    The current problem with the concept of grid computing is the lack of an easy way to deploy it in a standard business environment. What the article and its links are driving at is coming up with a cheap and easily implimented mechanism to turn every office, and chain of offices into a grid.

    In theory, you could sell your unused processor cycles the same way people who generate their own power sell power back to their power companies. You ISP could actually, someday become a processor cycle reseller and you could operate on a minimal set of hardare in the typical office enviroment becuase you can always pick up extra cycles from your ISP when you need them.

    Ah, the pipe dream.
  • I guess I would disagree that the Grid is where HTTP was circa 1993. Whereas there was just one WWW and it was based on one protocol, the present definition of Grid computing is hazier.

    There are several competing definitions of "Grid" going around - from the happy-big-cluster idea that Apple calls Xgrid (bad name, good product, IMHO) to the TeraGrid and NCSA grids in the US to the LCG/Grid3/Nordugrid to SETI@home. They all speak different languages and are built on different models.

    Most of the definiti
  • One of the points in this article is that many companies have idle computing power and there servers are under utilized. Obviously if there existing infrastructure is more than handling it's load they are not going to be too keen to cross over to this new technology (well it's not really a new idea).

    On the other hand it does mean that new networks can be created using less resources, but at the moment the biggest interest in this would probably be from the scientific community to do really intensive proces
  • Okay...I'm not completely up in the inner workings of GRID computing, but is the premise the same as those used in the past for other distributed environments such as DCE (Distributed Computing Environment ) [opengroup.org] or CORBA (Common Object Request Broker Architecture) [omg.org]?

    My experience with DCE at least was that it was a distributed environment that took a lot of coordination between systems, which unfortunately was not done very well in the environment I'm familiar with. As a result of this it did not prove robust e

    • Re:GRID = CORBA or DCE Repackaged (Score:1, Interesting)

      by Anonymous Coward
      How is GRID different from these methods?

      Grid is a set of capabilities which must satisfy requirements at a much higher conceptual level. Whereas we might say that DCE and CORBA define the atomic structure, Grid provides the biochemistry.

  • If grids worked, hosting companies would sell them (Score:3, Interesting)

    by Animats ( 122034 ) on Monday November 01, 2004 @12:28PM (#10687623) Homepage
    As I keep pointing out, if grid computing was good for anything, it would be a service that hosting companies sold to keep their machines busy during off-peak hours.

    Hosting companies have large numbers of identical machines with high bandwidth interconnects. That's just what you want for "grid computing". They're already set up to allow customers to run applications on their machines, and are able to deal with the security problems. Load is very low during off-peak hours. The machines stay up; they don't suddenly get disconnected from the net because somebody turned their desktop off. They're all loaded with the same base software. It's the ideal situation for commercial "grid computing".

    So why is nobody selling this? Because there's no market for it. There's no real commercial market for supercomputer time, distributed or otherwise. Once upon a time, from about 1960 to 1980, there were engineering computer service centers, where you bought time-sharing service on big mainframes. Control Data and UNIVAC were the preferred machines for this. But that business is dead. CPU time became too cheap.

    A well-known commercial grid was Gateway Processing on Demand [archive.org], announced in late 2002 with great fanfare. Gateway offered "grid computing" on thousands of Gateway-owned machines. They quietly dropped that service some time last spring. Their former CEO admitted that it generated "not a lot" of revenue. Basically, it was an attempt to generate some revenue from Gateway's unsold inventory of machines.

    Grid computing is one of those schemes where all the interest is on the sell side. Nobody wants to buy it. "Micropayments" and "portals" are like that. They didn't sell either.

Slashdot Top Deals

Do you suffer painful hallucination? -- Don Juan, cited by Carlos Casteneda

Close