Slashdot Log In
One Data Center To Rule Them All
Posted by
CmdrTaco
on Mon Sep 08, 2008 09:44 AM
from the my-preciousss dept.
from the my-preciousss dept.
1sockchuck writes "Weta Digital, the New Zealand studio that created the visual effects for the 'Lord of the Rings' movie trilogy, has launched a new "extreme density" data center to provide the computing horsepower to power its digital renderings. Weta is running four clusters that are each equipped with 156 of HP's new 2-in-1 blade servers, and use liquid cooling to manage the heat loads. The Weta render farms currently hold spots 219 through 222 on the current Top 500 list of the world's fastest supercomputers."
Related Stories
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Full
Abbreviated
Hidden
Loading... please wait.
Obligatory... (Score:4, Funny)
No, imagine (Score:5, Funny)
Parent
Re:No, imagine (Score:5, Funny)
Networked on a Tolkien ring?
Parent
Re: (Score:2, Funny)
Re:No, imagine (Score:4, Funny)
Yeah, that was quite preciousssssss.
Parent
Re: (Score:3, Informative)
Throughput on a Tolkien ring can reach very high performance levels when the data is encoded in an Elvish Language.
For strange characters support, unicorn encoding is recommended.
Re:No, imagine (Score:3, Funny)
a balrog cluster of them!
That would be a daemon of the ancient computers.
Re: (Score:2, Funny)
So, does Mithrandir then become the code name for the firewall, preventing pre-release leaks of footage?
"You cannot pass!!"
Re: (Score:2)
Releasing the heat? (Score:5, Funny)
Two Towers?
Re: (Score:3, Funny)
They don't, the Orc's working there, have been waiting for a hot water bath for years.
"Extreme Density" computing can be hazardous (Score:5, Funny)
I can't mention who I work for for obvious reasons, but we did some experimenting with "extreme density" computing some time ago as part of a black ops project for the government. We achieved densities previously unheard of by man.
Unfortunately, we got greedy. We increased the density so far that the entire facility ended up collapsing into a black hole, wiping out much of the state of North Dakota. We were able to contain the damage, and we've managed to keep it a secret by replacing the state with a hologram projection, but eventually someone is going to go there and figure out that something is amiss.
...and a darkness spread over the land... (Score:2)
Note the vague phonetic similarity between "North Dakota" and "Mordor."
Did this project happen to hand out t-shirts... or, hm, something less obvious like... rings?
Re: (Score:2)
wiping out much of the state of North Dakota. We were able to contain the damage, and we've managed to keep it a secret by replacing the state with a hologram projection, but eventually someone is going to go there and figure out that something is amiss.
Wiping out North Dakota [wikipedia.org] shouldn't be too hard. There's only about 600,000 people in the state [wikipedia.org] and half of them live on the eastern border within 4 miles of Minnesota.
Re: (Score:2)
Re: (Score:2)
That far north, it's amazing they have as many people as they do up there.
Wow--So What Was It Like? (Score:2)
Amazing! So what was North Dakota like before it became the vast, desolate wasteland devoid of any trace of humanity it is now?
Re: (Score:2)
There's an 'e' on the end of Gilmoure.
Re: (Score:2)
Heh. Always makes me laugh, hearing the name. I've been using Gilmoure since I started gaming, in late '70's. Means 'dark star' in Elvish.
Water-cooled datacenters (Score:5, Interesting)
I'm of a mixed mindset when it comes to water-cooled datacenters.
On one hand, you've got the makings of a biblical scale disaster with all that water and electricity mixing.
On the other hand, you can't argue with it's effectiveness.
I'll stick to non-catastrophic issues when my
air conditioner breaks down.
Re: (Score:2)
Re:Water-cooled datacenters (Score:5, Informative)
Liquid cooled, not water cooled. They cool it in an inert liquid, rather than water.
Parent
Re: (Score:2)
Ouch. I've seen the prices of some of the 3M inert liquids, and those are incredibly expensive. I can't imagine buying the hundreds of gallons it would take to cool an entire datacenter.
Also, like Critical Facilities above mentioned, what do you do for heat dissipation in the event of a leak?
Re: (Score:2, Informative)
I can't speak for anyone else, but in our data center we have two sets of chilled water pipes and two sets of return pipes, in case one does break. They are valved all over the place so any one set of pipe can go down and the others continue working.
As well, the chilled water never enters the data center. Our CRAC units sit outside of the data center and are ducted in overhead (because we can reach higher space velocities than with a raised floor). Thus, no chilled water ever gets inside of hte data cent
Re: (Score:2)
From the fine article:
"That kind of density creates a cooling challenge. âoeOne of our most important decisions was to invest in water-cooled racks from Rittal, which allows us to precisely control the amount of cooling that a specific rack requires,â said Shand."
Re: (Score:2)
Pure water is electrically inert. Though given how dusty servers tend to get, I doubt it really matters.
Regardless, I've never had a watercooling leak that wasn't my own fault (not connecting the fitting properly), and obviously in this kind of scenario you test things first. Then again, I'm talking about a couple of boxes, not a datacenter.
Fluorinert or something similar would be very effective and a whole hell of a lot safer for this. It's certainly not cheap, but given the fact that you can effectivel
Re: (Score:2)
it's only inert until it is mixed with some dirt/dust/grime/paint. :(
Re:Water-cooled datacenters (Score:4, Informative)
Parent
Re: (Score:2)
Throw in some bikini-clad babes and you've got a supercomputer cluster that makes its own porn!
Re:Water-cooled datacenters (Score:5, Interesting)
I'm of a mixed mindset when it comes to water-cooled datacenters.
On one hand, you've got the makings of a biblical scale disaster with all that water and electricity mixing.
We had such a disaster many years ago.
The coolant lines themselves circled the walls near the ceiling of the dinosaur pen. Beneath each line was a drip tray that was alarmed to sound in case moisture was detected. These drip trays ran the entire length of the coolant lines. That is, the entire length except for about six inches in the very corner, where it was too hard for the tray installers to get to because they were behind the conduit leading down to the main power transformer. I can only assume that all that conduit also made it difficult for the plumbers to properly solder the elbow in the cooling line as well.
Of course there was only one place for the pipe to leak. The version of the tale I heard implies an impressive display of fireworks was seen by all present as the coolant entered the transformer.
Fortunately, the loss of power did not damage the mainframes (except for their ability to run.) The rooftop generator was fired up, and in short order the mainframes were back on line.
And in short order the generator engines stopped because the fuel tanks were kept almost empty, the plan being to fully fuel them only in case of need.
Once the replacement fuel was delivered, the generators ran for only a few minutes before dying again, this time for good. The ancient fuel had congealed in the injectors once the engines had stopped running.
We ran our data center for two weeks powered from a truck-based generator parked in the street while the electricians replaced the transformers and repaired the generators.
I now hear the diesel generator being started every month or two, and run for a few hours.
Rather than a biblical disaster, I'd say it was more like a Marx Brothers' movie.
Parent
Re: (Score:2)
wow, that's rough, but thanks for sharing!
Our generator test-runs every Thursday for a few minutes to make sure everything will be ok. You know, theoretically
It doesn't always work out like that [blogspot.com]
Re: (Score:2)
Now that you mention it, I've actually done that. I discovered (the hard way) that mineral deposits built up inside the valve get dislodged when it's triggered, and sometimes those chunks then get in the way of sealing the valve again. Which meant a stream of hot water that couldn't be shut off until I shut off the house valve and couldn't be turned on again until I bought a new pressure relief valve.
So now I don't do that anymore.
Re: (Score:3, Informative)
Well they may be using a different liquid than water. Mainframes have been using water cooling for years.
But even if they are using water we are talking about professionals here. They have been water cooling electrical devices that are far more dangerous than CPUs for many years.
http://www.cobermuegge.com/details.asp?id=90 [cobermuegge.com]
And here http://en.wikipedia.org/wiki/Vacuum_tube#Cooling [wikipedia.org]
Re: (Score:2)
Those are cool links, thanks. I hadn't seen them before.
Liquid Cooled! Awesome (Score:5, Interesting)
I think it's great that they are using liquid coolants for their system. Whenever I see a traditional server farm, I just can't help but think that air conditioners are so inefficient for the task of cooling computers. Not only do you have to cool the air, you also have to blow it around. The floors in some data centers are raised just to allow better airflow. And if you think about it, only the insides of the computers have to be cooled, not the entire freaking room. I hope this ushers in a new age of more power-efficient computing.
I also think it's pretty funny that a supercomputer is used to make movies.
Re: (Score:2)
I know what you mean, but I get nervous about that much water around my computers. A leak would be catastrophic.
When my air conditioner breaks down [blogspot.com], I don't have a life threatening situation.
That darn Vinge... (Score:4, Informative)
I also think it's pretty funny that a supercomputer is used to make movies.
It was pretty funny forty years ago [wikipedia.org], too.
Parent
Re: (Score:2)
Re: (Score:2)
The Rittal racks just have a coil in them; the servers are cooled with air.
On a power basis, there isn't much of an efficiency improvement. Your biggest gain is if you take the filters out since the cabinet is sealed.
As for a leak hitting the computers, the coil is in a sidecar which is designed pretty well to segregate leaks. Biggest concern is usually what happens when you lose water or fans. Most of the cabinets open the doors automatically.
And mega bill for bandwidth? (Score:5, Interesting)
I wonder how much they have to pay for external bandwidth. I always thought that "super data centers" are used to help split the job between multiple special effects studios, so, say, group in London can work on part of the shot and still have all data in the same place.
Except in New Zealand there are no "unlimited" plans, and there are severe bandwidth caps in place.
Re:And mega bill for bandwidth? (Score:5, Insightful)
Parent
Re: (Score:2)
Leased OC192 lines are probably available there. I'm willing to bet they're not on "broadband" too.
Re: (Score:2)
I'm kind of surprised. It doesn't look [cnet.com] like NZ has any major bandwidth, compared to the rest of the world. Maybe they can get Google to hook them up [blogspot.com] to the pan-Asian cable going in soon. ;-)
Water cooling at CERN (Score:2, Informative)
Hold's spots 219-222... (Score:2)
The Weta render farms currently hold spots 219 through 222 on the current Top 500 list of the world's fastest supercomputers.
That's a lot of orcs.
Re: (Score:2)
...will still pale in comparison to ILM. LOTR had some of the worst compositing I had ever seen. The fact that ROTK won an Oscar for Visual Effects only further illustrates the irrelevance of the politics behind the Oscars.
I don't know, I don't think ILM did a great job with Naboo or Tattooine in episodes 1-3, either. The battle of the Gungans vs the robots was a giant load of CGI all around. And that was released in 2005, while ROTK was released in 2003.
But yeah, Gollum was too aggressive for Weta, at least when they filmed it. And although the character was certainly more repulsive, Jar-jar was better done than Gollum (slightly.)
Maybe the new render farm will let them try some better algorithms or more complex mode
Re: (Score:3, Insightful)
Or you're just a film snob that disagrees with the majority of normal people who actually liked the movie because it looked good and was entertaining.
And why is it that people always have to look down their noses at various items that do well in the general public? It's like if it's popular and the unwashed masses salivate over it, some jackass has to show how superior they are by being different and telling the world that those unwashed masses really are in fact, unwashed. And probably drooling or somethin
Re: (Score:3, Interesting)
You're either a true expert in the field, who does film-grade compositing every day, or you're a troll.
I've done some chroma-key compositing over background plates myself, albeit for TV. I can honestly say I did not see any artifacts in LoTR - although I admit I haven't checked frame-by-frame.
Either way, show some examples if you want to be taken seriously.
Re: (Score:2, Insightful)
Here's a hint: It's not what you said, but how you said it.
Re: (Score:3, Funny)