Google's Secret Plans For All That Dark Fiber? 534
beat.net writes "Robert X. Cringely details the plan for all the dark fiber Google has been buying up: "The probable answer lies in one of Google's underground parking garages in Mountain View. There, in a secret area off-limits even to regular GoogleFolk, is a shipping container. But it isn't just any shipping container. This shipping container is a prototype data center. Google hired a pair of very bright industrial designers to figure out how to cram the greatest number of CPUs, the most storage, memory and power support into a 20- or 40-foot box. We're talking about 5000 Opteron processors and 3.5 petabytes of disk storage that can be dropped-off overnight by a tractor-trailer rig. The idea is to plant one of these puppies anywhere Google owns access to fiber, basically turning the entire Internet into a giant processing and storage grid. While Google could put these containers anywhere, it makes the most sense to place them at Internet peering points, of which there are about 300 worldwide.""
Re:Nice work of fiction (Score:5, Informative)
I used to work at a datacenter and we had a generator small enough that you could fit 12 of them in a shipping container, and the genny was enough to run a 500 machine datacenter for three days without refueling. The portable datacenter may well have a generator included.
Salt (Score:5, Informative)
Saying that, when it comes to technology at least, he is speculative is something of an understatement. Take what he says with an extremely large grain of salt.
Re:5000 Opterons (Score:3, Informative)
Re:The Alternative Re:Google is Skynet? (Score:3, Informative)
Re:Akamai (Score:2, Informative)
Actually, the three Akamai's we had are still running right on top of the server I administrate there.
I too was never quite clear on what they did, but my understanding is that they provide local cached copies of oft-visisted webpages. Even Google--last I knew, that is.
So it's more like, instead of having to access Apple servers every time I type "Apple.com", I'll actually be accessing the local version hosted at my ISP on their Akamai box.
In a way, Google already provides this with Google cache. So it really wouldn't be a big step for them to do it real time.
--Petey
Re:article doesn't explain network (Score:5, Informative)
To remove the network latency between them and you.
They're not being used "for computing" in the sense you're envisioning. For one thing, 5000 Opterons is enough to tackle pretty much any problem you'd care to throw at it, so there's no need to talk to anyone else. For another thing, they wouldn't be doing big computations, they'd be doing massive numbers of small ones. Think Gmail. 3.5PB is enough to store an awful lot of email, and a few thousand Opterons can run rather a lot of simultaneous HTTP connections from people accessing the mail. Add in a fast network link (for talking to all those many people accessing the mail, and for replicating everything offsite), and you're set.
Cringeley's penchant for sensationalism aside, it's pretty clear that Google's got the expertise and the mindset to deal with problems that start with "if we had 10,000 fast CPUs, 10,000 hard disks, and 10,000 GB of RAM...". Google's rapidly expanding, and has been ever since they started. Back when Google fit in a closet, a new server constituted a big expansion. I'm not surprised that these days their unit of expansion is a tractor trailer with a few dozen racks in it. And if you've got something that packages up that nicely, it only makes sense to pepper the globe with capacity.
Your numbers are off... (Score:4, Informative)
(40 * 8) * 4 + (8 * 8) * 2 ==
1408 sq. ft.
which, for 1 megawatt, is more like 710 watts/ sq. ft.
Re:Lots of heat, lots of power (Score:5, Informative)
The sort of temperature-differential energy recovery you speak of is technically possible but isn't efficient enough to substantially reduce the cluster's power requirements, and thus its need to vent waste heat.
Re:Cooling 5000 Opterons? (Score:3, Informative)
Not all that much of a heat load to deal with (Score:3, Informative)
Clearly you're talking about serious energy density here with cooling which is on the order of what it took to cool a 637 class nuclear submarine underway in moderately cool water. Of course you'd really need TWO of those a/c units because you will want to protect that investment (still fits in the same footprint) and be ready to pay some substantial utility bills.
Re:additionally... (Score:2, Informative)
So spare us your oh so late lecture from 2001, we all know that peering point are not just MAE-WEST or EAST, you did right???
Already been done by the telcos... (Score:2, Informative)
Re:Not all that much of a heat load to deal with (Score:2, Informative)
If I were engineering it, I'd probably skip the water step and investigate CO2 or freon refridgeration cooling. CO2 can be used as a refridgerant in place of freon, but it's typically not as efficient for air cooling purposes. The only real reason to look at CO2 is that it may be a safer solution when you have so many possible sources of leaks. Presumably you'd have multiple loops and leak detection/shutoff valves on each blade. You'd have to be careful to make sure that any condensation from your blade or chip coolers drips into a drain system. You could also put a thermal plug in each blade and use it for primary fire supression. Precautions against leaks would still be necessary, as high concentrations of either gas can kill people.
Re:aren't they all? (Score:2, Informative)
Re:Is this a joke? (Score:4, Informative)
Re:Lots of heat, lots of power (Score:3, Informative)
As for lots of hot water - if you are worried about heating up part of a small lake you just run the water through some open drains to cool off a bit first. If you are going to recirculate it you let it drop down from a height to get some evaporative cooling - the principle behind cooling towers.
Crmblznski's Limit -- Definition (Score:4, Informative)
Crmblznski's Limit, sometimes spelled Crizmblski's Limit, has its origins in Keith Laumer's novel "The Great Time Machine Hoax" [1].
The basic theorem is that there is a finite limit to the complexity of any given machine, which specifically precludes the operation of "a machine with sufficiently extensive memory banks, adequately cross-connected and supplied with a vast store of data, [that by its very essence] would be capable of performing prodigious intellectual feats simply by discovering and exploring relationships among apparently unrelated facts." The Limit is an irrational number, much like Pi, in that the total complexity of machine is wholy dependent upon both hardware and software designs.
Re:Is this a joke? (Score:3, Informative)
Here's a link to the entire text of the novel, for anyone that's interested:
The Great Time Machine Hoax [webscription.net]