Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 


Forgot your password?
Close
typodupeerror
×
Technology

Weta Digital's Render Farm Upgrade 313

Posted by CmdrTaco from the never-before-have-I-wanted-to-lick-a-wall dept.
Headspace2 writes "Weta Digital (The graphics company behing LOtR computer effects) has just purchased 220 2.2GHz dual Xenon machines, each with 4GB of ram, to add to their current render wall of 350 1 Ghz P3 systems. They have also placed an order for another 256 Xenon servers. And it's all running Linux. My favorite quote is 'it is thought the server farm will be the most powerful processing site in the Southern Hemisphere'. They should use that in the FotR ad campaign... 'Rendered using the most powerful processing site in the southern hemisphere' Congrats the guys that get to play with all those clock cycles. Make more movies.
This discussion has been archived. No new comments can be posted.

Weta Digital's Render Farm Upgrade More

Weta Digital's Render Farm Upgrade

Comments Filter:

  • Xenon? (Score:5, Funny)

    by syates21 ( 78378 ) on Tuesday July 16, 2002 @09:51PM (#3899027)
    Powered by noble gas. Woohoo!!
  • Wow. I wonder what their Distributed.net keyrate would be?

  • *sigh* (Score:5, Informative)

    by ottffssent ( 18387 ) on Tuesday July 16, 2002 @09:53PM (#3899032)
    Xenon is an element. Xeon is an expensive CPU. I see "Intel Xenon" too many times at work. Please not on Slashdot too.
    • Maybe this is a reference to a vaporware CPU offering from Intel?
    • "Xenon is an element. Xeon is an expensive CPU. I see "Intel Xenon" too many times at work. Please not on Slashdot too"

      It's really funny how people posting on a public forum act like it's really painful to see bad spelling and grammar.

      • Re:*sigh* (Score:2, Insightful)

        by jerde ( 23294 )
        It's really funny how people posting on a public forum act like it's really painful to see bad spelling and grammar.
        But when isn't painful? I see it as exactly analogous to hearing a note out of tune.

        I think his comment was that the actual stories on /. should be edited for correctness. Isn't that what the editors are for?

        - Peter
        • I think it's fine to expect that from the article itself. To expect that in the summary that's submitted by individuals is akin to expecting it from from every single person who comments on Slashdot.

          If ya understand them, don't wince. I can understand the inchs vs. centimeters thing earlier, but Xeon vs. Xenon isn't that ambiguous.

      • Re:*sigh* (Score:4, Insightful)

        by Chasuk ( 62477 ) <chasuk@gmail.com> on Wednesday July 17, 2002 @12:29AM (#3899596)
        I do find it painful. Not in a literal sense, of course, but I cringe in the same way that I might if I were addressed by a drunk standing too close to me, reeking of halitosis, with snot dripping from his mustache.

        No, I'm not trolling, this isn't flamebait, and I'm not being elitist. I'm just pointing out that some readers do experience a visceral response to poor spelling and grammar.

        Grammar doesn't have to be perfect, or I would never post. Spelling is a nearly impossible chore for some: it is acceptable, for them, if dyslexia or a similar disorder is their excuse. However, poor spelling and grammar, if due to laziness or indifference, does offend me.

        Further, from experience, I have seldom read a thought worth reading that was contained within a syntactical nightmare.

        I've been reading Slashdot for years, and I have noticed that the literacy levels - and levels of intelligibility and thoughtfulness - have declined as it has become a destination visited by more people.

        Has anyone else noticed this deterioration? It has gotten so bad that I'm now reading www.kuo5hin.org more often than Slashdot.

        Now that this message has rambled entirely off-topic, can anyone recommend intelligent, literate forums with a high volume of traffic? They _don't_ have to be tech-oriented.

        All suggestions welcome.

        • Re:*sigh* (Score:2, Informative)

          by Citizen of Earth ( 569446 )
          Grammar nazi mode activated...

          They _don't_ have to be tech-oriented.

          You shouldn't use the word "have" in this way. "Have" is a verb that means to possess something, so you are saying "They _don't_ possess to be tech-oriented". Instead, say "they don't *need* to be tech-oriented" or "they aren't required to be tech-oriented". (Yes, I put my periods outside of literal quotes; what's your problem, buddy!)

          And while I'm at it, it also bugs me the way that people say "different than". It's "different from"! The former is like saying "compared than", which doesn't make any sense.
          • While we're being petty, you might check a dictionary to back yourself up. A nice thick one, since the definition of "have" won't support you, so the book might as well. Have, in the sense the above poster was using it, is an accessory or helper verb.

  • Don't you mean... (Score:2, Informative)

    by nemesisj ( 305482 )
    Dual Xeon ?
    • From the article:
      The machines are rack-mounted dual Xeon-processor systems operating at 2.2GHz with four gigabytes of memory each. The 950 processors will be added to 350 existing 1GHz Pentium 3 systems as part of a dedicated "render wall" comprising 22 racks.
      Now, see, if they could get xenon doing anything useful in a CPU, I would be impressed.

  • Comment removed (Score:5, Funny)

    by account_deleted ( 4530225 ) on Tuesday July 16, 2002 @09:54PM (#3899041)
    Comment removed based on user account deletion
  • Wow. It could only be more politically correct if the chips were Athlon MPs.
  • could they also claim that it is the most powerful processing site in the eastern hemisphere?
    • I don't think so. Isn't that big new weather-prediction site in Japan? I would imagine that's bigger, though I haven't checked the numbers.

  • I bet everybody... (Score:1, Funny)

    by Anonymous Coward
    ...will correct you on "Xenon" while completely missing "behing".

    Typical Slashdot.

  • 100Gbps? (Score:2, Funny)

    by willith ( 218835 )
    "The servers run in parallel and major jobs are broken down for each server. It is networked together with 100Gbps ethernet and Foundry networking switches...."

    A hundred gigabits per second? Dude! Sign me the hell up!

  • About those Xenon wisecracks... (Score:5, Funny)

    by Devil's BSD ( 562630 ) on Tuesday July 16, 2002 @09:57PM (#3899054) Homepage
    You could use xenon to power a quantum computer. Dual xenon = 2 xenon atoms = 2 qubits, which could be roughly 64 bits, or the processing potential of a potato.
    • How do you count qubits?

      I'm pretty sure that there's no direct equivalence formula between qubits and bits...
      • Actually, iirc 2 qbits is the equivilent to 4 bits, as qbits can be 1 and 0 at the same time. Two regular bits = 2(bits)^1(#of states a bit can be); however with qbits = 2(bits)^2(#states).
      • And Noah said, "God...what's a qubit?"

      • Re:2qb == 64b ? (Score:2, Informative)

        by Fyndlorn ( 88381 )
        There isn't...

        Just think of it in terms of hilbert spaces (or just plain vector spaces). A qubit spans a 2 dimentional hilbert space. A (normalized) state on that space could be written

        |S>=a1|0>+a2|1>

        just think of |> as a vector, where a1 and a2 are ANY complex numbers such that |a1|^2+|a2|^2=1

        for two qubits then you just have a 4 state space

        |S>=a1|00>+a2|01>+...

        for more info check out:
        http://search.barnesandnoble.com/textbooks/b ooksea rch/isbnInquiry.asp?userid=66NT518KIO&isbn=0521635 039

  • Do I see some movement ... (Score:5, Funny)

    by Nostrada ( 208820 ) on Tuesday July 16, 2002 @09:58PM (#3899058)
    ... in some of the decentralized computing efforts, coming from the southern hemisphere?

    Team LotR strikes at Distributed Folding, ECCP, Folding@Home, Genome@Home, OGR (24 and 25), RC5, Sengent D2OL, SETI, UD ...
  • ... a Beowulf cluster of.. waitaminnit. It probably *is* a Beowulf cluster. :-/
    No, I haven't read the article. Is that necessary?

  • So much power... (Score:5, Funny)

    by Toasty16 ( 586358 ) on Tuesday July 16, 2002 @10:01PM (#3899063) Homepage
    And yet they still can't make Frodo look like a guy.
  • Since there are all kinds of benchmarks, which would be most appropriate here? I'm thinking heavy fpu performance. Most gaming sites only compare single processor performance. Can anyone dig up a benchmark of a dual P4 of 2.2GHz or thereabouts? Then compare to the P3s. I'm guessing this more than doubled their capacity.
  • ...without a front-page type or two? Xenon machines, FotR... I understand typos happen everywhere, but when you're putting out a product like slashdot.org, even asking people for money for ad-free browsing, you would think you could expect some basic editing of the stories. Is it that much to ask to have some one read over the story once or twice before it's posted?

    Out of 10 or 12 stories a day, there are always one or two with bad grammar and/or spelling. This definately takes away from any sense of professionalism slashdot.org presents on itself. Consider this editors, everyday this website is your best resume`. You wouldn't submit a resume` that has grammatical errors on it, would you?

    • Rules for flaming based on spelling/grammar (Score:4, Insightful)

      by hayden ( 9724 ) on Tuesday July 16, 2002 @10:14PM (#3899122)
      Make damn sure you don't make the same mistakes.

      ...without a front-page type or two?

      "typo" maybe?

      Is it that much to ask ...

      "Is that too much to ask ..." or possibly "Is it too much to ask ..." depending on what you wanted to do with the rest of the sentence.

      ... of professionalism slashdot.org presents on itself.

      Slashdot and professionalim in the same sentence has to be some sort of error.

      You wouldn't submit a resume` that has grammatical errors on it, would you?

      Surely you mean "in it".

      My point? Enough with the bitching about the spelling/grammar. Most of people here aren't any better and of the remainder most don't care.

  • Weta Digital's Render Farm Upgrade

    is it the "Render Farm" of Weta Digital that is being upgraded?

    My head hurts...
  • Any chance they could use those extra cycles for cracking RC5 blocks? Give the slashdot.org team a run for their money, eh?
  • I wouldn't say make more movies, I'd say make better movies.
    • using the infinite monkeys with infinite word-processors theory, I'd have to say that make more movies is the way to go. Would have a much better chance of a monkey pounding out a better movie than the throw a script together crap we see these days.

  • Great marketing idea! (Score:5, Interesting)

    by DearSlashdot ( 592493 ) on Tuesday July 16, 2002 @10:03PM (#3899074)
    They should use that in the FotR ad campaign... 'Rendered using the most powerful processing site in the southern hemisphere.'

    Sure, who cares about plot or character development? We've got a server farm!

    Who submitted this? George Lucas?

    • "Sure, who cares about plot or character development? We've got a server farm!"

      Normally I'd agree with you, but since it's based on a book all that stuff's accounted for. At least we know that they have the CPU horsepower to render out some wild stuff.

      Personally, though, I'd rather know that they hired super-talented animators. Them'z worth heaps more than the most powerful render farm.
  • I wonder, with todays CPUs becoming more and more powerful, won't the actual internal architecture and the standard data transmission system (copper or gold or whatever those conduits are made of) end up outdated and not being able to cope with the CPU's speed? Maybe this applies to other pieces of hardware as well? HDs not being able to feed procs the data they require fast enough or RAM suffering from aforementioned problem...

    How do they solve these kind of issues anyways, especially in extremely large computer arrays like this one mentioned here but also in supercomputers or maybe even in our home PCs which are getting faster and faster as well every few months...

    • Re:CPU vs data transmission speeds. (Score:5, Informative)

      by Qrlx ( 258924 ) on Tuesday July 16, 2002 @10:21PM (#3899151) Homepage Journal
      Maybe you haven't noticed, but bus speeds have increased since you bought that Pentium 75 system. Though not as much as cpu speed, because that's historically been the focus of the, uh, personal computer. Who cares about optimizing network transactions on a PC? They were built to get away from mainframes, remember? Well, that was true 20 years ago and the paradigm has stuck for longer than it should have.

      Even so, Consumer hard drives can now claim ATA-133 speeds, that's probably an order of magnitude faster than the 1.2 GB drives from five years ago. And SerialATA is coming. On the server side, I think U320 SCSI is out now. SCSI started at 5, now it's at 320. THat's like 64 times faster.

      RAM has kept up, too. The first DIMMS were 66MHz, now you can get effectively 400MHZ DDR, or faster than that if you want soon-to-be-out-of-business RAMBUS.

      Heck they invented the AGP port so we could play games, and that's at 4X now, with 8X on the horizon and some really bigtime advances in GPU power in just the past two years.

      None of these have seen the speed increases of the CPU, but they are moving along at a nice clip. The PCI bus is maybe the weakest link here, but it's gotten better.

      I think there's a lot of room for growth left in the current physical materials. I keep hearing 15 years until we hit the quantum barrier in CPUs, if we keep up with Moore's Law. There was a great article not so long ago about hard drives, and how they are basically doubling in areal storage density every year. In ten years, you can get a 120 Terabyte drive. Only one problem: What the hell would you put on it to fill it up?

      Kinda like the predicament they find with broadband. There's nothing else to do with all that bandwidth than download mp3s and pr0n and warez. Oops.

    • There's always a bottleneck somewhere. It's been the drives, the bus, the expansion slots, the network, the ram...

      Our biggest issue right now (in my mind, anyway) is physical media. Sure, ATA 133 is burstable to 133, but who actually thinks they'll get 133 for any length of time. If you Cause Win98 to hang at the End Task screen, the buffer on the drive might fill up and you could get maybe a half-second of 133.

      The only way to get great speeds out of media is RAID striping or other such technologies.

      Don't know if the cluster they have set up uses much (decentralized) storage, but the network has got to be huge.

  • $5 bucks says they just want be able to play Doom III at full detail ;)

  • Who gives a monkey's chuff? (Score:4, Funny)

    by Beatlebum ( 213957 ) on Tuesday July 16, 2002 @10:08PM (#3899098)
    >> They should use that in the FotR ad campaign... 'Rendered using the most powerful processing site in the southern hemisphere'

    So that morons like Taco can point this out to their long-suffering girlfriends?

    Who gives a fuck. Seriously dude, get a hold of youself and try not to be a weiner all your life.
  • The cost of the 476 machines is pegged at between $3M and $4M. Assuming $3.5M (midpoint), it works out to $7352 per machine; assuming NZDs, that comes to about USD 3564 per machine. Since no HDD is mentioned, they most probably do no have any.

    Assuming all the assumptions above were correct, how does the cost compare to something comparable stateside? Of course, I'm ignoring the "100Gbps" network(!) and the Foundry switches, but I don't think they'll add more than a couple of hundred bucks per machine _at most_.

    • Is it the most powerful render farm in all of Middle Earth?
    • Is this the Matrix of Middle Earth? I've always wondered about that Agent Guy.
    • Imagine a beowulf cluster of these ...

  • They should use that in the FotR ad campaign... 'Rendered using the most powerful processing site in the southern hemisphere'

    Pshaw! That'll only appeal to geeks! Oh, wait a minute...

  • I bet they get killer frame-rates!!
  • I wonder if these guys are actually getting the most bang for the buck. Sure, they are buying the fastest machines, but I sure wonder if a cluster of 300 Pentium 4 2.0 GHz or even Athlon 1900+ wouldn't be faster. According to mwave [mwave.com], the Athlon MP 1900+ currently sells for $192, while the P4 Xeon 2.2 sells for $304. Everything else being the same, that's $100 per box saved.

    Assuming a base platform cost (without processor) of $400 for MoBo, memory etc., the P4 Xeon would have to be 17% faster than the Athlon to justify the premium. According to the benchmarks on Tom's Hardware [tomshardware.com], Intel would have a hard time attaining that.

    Jan

    • Assuming a base platform cost (without processor) of $400 for MoBo, memory etc., the P4 Xeon would have to be 17% faster than the Athlon to justify the premium. According to the benchmarks on Tom's Hardware, Intel would have a hard time attaining that. You may be right but those benchmarks don't support your case much because they don't include any P4 Xeons.

    • Re:Bang for the buck (Score:2, Informative)

      by psamuels ( 64397 )
      I wonder if these guys are actually getting the most bang for the buck. Sure, they are buying the fastest machines, but I sure wonder if a cluster of 300 Pentium 4 2.0 GHz or even Athlon 1900+ wouldn't be faster.

      I agree with you, but to play Devil's Advocate, there are sometimes reasons you want a fast CPU, not just a fast cluster. Our SGI sales guy often tries to make this point, for the obvious reasons, but it's true.

      If you are rendering out a large number of frames, you want the most possible aggregate CPU power, because rendering is extremely parallelisable (each frame stands alone). So 50 Athlons is better than 40 Xeons. But if you are just rendering out a 5- or 10-frame test sequence, and the wall is not already overloaded - then you want the 40 Xeons instead, since each one can take a frame and you'll get your result back faster.

      There is also the issue of network bandwidth. In some cases you can benefit quite a bit by having fast boxes with as many CPUs per box as possible. This is because there is a non-trivial network burden in sending out the job to be rendered, along with all its textures, images, etc. This can be mitigated by multicasting and caching - but I don't know to what extent Renderman does this - but I know if you don't design it right, it can really slow down your jobs. (We evaluated one render distribution system that relied on Windows SMB file sharing for its I/O. Sending a 200-MB job to each of 10 render crunchers pretty much killed it.)

  • on THEIR SETI@home team!

  • Make more movies? (Score:2, Interesting)

    by Milinar ( 176767 )
    Maybe, maybe not. It's my impression that studios like this put tremendous amounts of capital into individual projects. They may very well not do another movie after the third LotR. Case in point: digital domain hired over 1000(!) animators to work on titanic - just imagine the amount of hardware they got. I could be wrong, but they haven't done a project nearly like that since.

    Plus, with moore's law, those machines they bought won't be worth the electric bill in a few years.
    -Milinar

    • Cameron was the most visible co-owner of Digital Domain, but he was actually a junior partner. WETA is, as far as I know, owned by Peter Jackson. He has a slate of films ready to roll. He doesn't have the dependence on "stars" like Arnie that Cameron has. He's on the opposite side of the world from Hollywood, and has produced a very profitable film. He has more freedom to make any film he likes than any other film-maker in years.

      Peter, who started working on effects in his mother's kitchen will be making use of this new toy for quite some time.

    • I wouldn't call $4 million NZ (or even US) to be a tremendous amount of capital, for a project like this. If they never use it for another project, they will still have gotten their money out of it. Sure, its a lot of money to me, but I bet they spent more than that on film stock.

      But I think the other guy is right and Peter Jackson will make good use of this equipment and these people in the future.

  • And for another "wise" Xenon joke... (Score:5, Funny)

    by vanza ( 125693 ) on Tuesday July 16, 2002 @10:48PM (#3899233)
    Talk about vaporware!

  • It is networked together with 100Gbps ethernet and Foundry networking switches.

    *sigh* My puny Netgear 100Mbps switch is feeling quite inadequate right now.

  • what graphics? opengl? (Score:2, Interesting)

    by horster ( 516139 )
    do these render farms use any graphics chips or are they done entirely in cpu?
    the reason I ask is that linux does not have any high quality open source opengl that supports the latest graphics boards. nvidia is probably the best for opengl support but not opensource.
    • do these render farms use any graphics chips or are they done entirely in cpu?

      Just the CPU. You want good floating point support [which is why Titanic used 500 dual-Alpha boxes], and memory bandwidth, and of course lots of Hz are always nice.

      Theoretically a renderer could use a GPU for a coprocessor, but I believe render software is so complex that any GPU on the market today would be too specialised to be of much use. Hardware acceleration works for games because the game developer can tailor the rendering requirements / algorithms to the capabilities of the hardware (as abstracted by OpenGL and Direct3D, or via vendor extensions to same). Render software, OTOH, is at the mercy of what the modeler / animator / compositor wants, and they are not willing to settle for "whatever the hardware can do the fastest".

  • ... what happens to these fine machines when they are retired, or when the studio deems them too slow?

    I sure can use one of these, gee, 4GB of RAM, that's more than the entire HD on my current machine.

    Ok, don't tell me to go buy a new one. My machine, as old as it is, it running Linux just fine, thank you. Has been serving me for almost 5 years, and 3 or 3 more years, than I'll consider... hehe.

  • Imagine... (Score:4, Funny)

    by swf ( 129638 ) on Tuesday July 16, 2002 @10:57PM (#3899269)
    a single one of these!!!!

  • In your face! (Score:3, Funny)

    by nihilogos ( 87025 ) on Tuesday July 16, 2002 @10:58PM (#3899275)
    The previous "largest server farm in the Southern Hemisphere" was in Tonga where 7 486s could render a scene from Tribes 2 in less than 17 minutes.

    So suck on that Tonga. And you never had the first dawn of the new millenium either.

  • Upgrading farms? Somebody call Blizzard... (Score:4, Insightful)

    by WIAKywbfatw ( 307557 ) on Tuesday July 16, 2002 @11:02PM (#3899288) Journal
    Gee, that's one unit upgrade that Blizzard obviously forgot to include in Warcraft III.

    A plough here, a grain store there, and voila, +50% to your food output. I'm surprised that nobody's thought of it before...
  • Scotty: "I just can't do it captain! I don't have the power!"

    "It's not the size of the render farm, it's how you use it."

    And of course, let's all imagine a Beowulf cluster of... oh. wait. Right.

    (Obligatory. Didn't say it was funny) :)

  • 100Gbps ethernet (Score:2, Informative)

    by dhammabum ( 190105 )
    Accoring to the article, It is networked together with 100Gbps ethernet and Foundry networking switches. 100Mbit perhaps?

    I looked on the Foundry website, 'only' 10Gbit.

    I hate those exponential powers!

  • wow... (Score:2)

    by M@T ( 10268 )

    I've always wanted to use Blender's "Render daemon" button...

    Seriously though, does anyone know what kind of modelling and render tools these guys are using ?

  • Oasis (Score:2, Funny)

    by terry_dyne ( 525072 )
    And after all
    You're my renderwall....

    Thank you Chicago -- Goodnight!
  • Yah! It was gonna be called Freon!

    Strangely, the idea got a very chilly reception, though everyone complained when it eventually got banned.
  • Everytime I read an article like this, it ticks me off.

    If they can get this sort of application running on a Linux system, why can't dreamweaver and Adobe port their products to Linux.

    Hell and damnation.

    I mean it can't be that hard.

    I guess the reason is because noone would buy the ports.

    Is Linux ultimately only useful to the custom solution and server crowd. Will the professional and consumer desktop ever be tamed?
    • Adobe made photoshop 3.1 for UNIX.....my university still has it if you are running an x-server and telnet in.

      so....maybee they will do it again....granted they are one of the biggest supporters of the BSA and might be afraid of the 'hacker' community
      • But noone would require them to realease the source.

        admittedly, they would have headaches porting binaries for the various distros.

        Maybe this is the thinking behind United Linux: make it easier to distribute binaries, not source that requires compiling and gives up intellectual property.

  • Massey [massey.ac.nz] university just announced [computerworld.co.nz] that it is going to build a 128 node beowulf cluster (no imagination necessary!). Auckland University [auckland.ac.nz] have recently got an IBM Regatta class machine.

    Just a (quite impressive) stone's throw away from Weta is NIWA's [niwa.co.nz] Cray T3E
    bash-2.03$ uname -a
    sn6908 kupe 2.0.5.51 unicosmk CRAY T3E

    I love running that uname :-)

  • Because if it would/could have been, they would probably have gone for a dual AMD system.

    You pay a lot more per XEON CPU, you pay a bit more for RAM (and the bit more adds up pretty quickly with 200 machines with 4GB), you pay a LOT more for the motherboard. I've had do to a renderfarm with budget restrictions, I got twice as much machines for the same price if that intel-based setup (and almost twice as much power).

    Stability? not any unusual issues that I wouldn't have got also with Intel-based stuff. I bought TYAN TIGER MPs, with dual athlon XP, and the hardware is top notch. The only issue I could see is if everything is heavily optimized for SSE2 and money isn't a problem, then it would make sense to grab P4 XEONs, but that's the only case I'd see (aside from marketting or direct rebates from intel for free exposure, etc etc) that could make someone take such a decision.

    My 0.02 cents.

Slashdot Top Deals

Don't panic.

Close