Google Reveals "Secret" Server Designs 386
Hugh Pickens writes "Most companies buy servers from the likes of Dell, Hewlett-Packard, IBM or Sun Microsystems, but Google, which has hundreds of thousands of servers and considers running them part of its core expertise, designs and builds its own. For the first time, Google revealed the hardware at the core of its Internet might at a conference this week about data center efficiency. Google's big surprise: each server has its own 12-volt battery to supply power if there's a problem with the main source of electricity. 'This is much cheaper than huge centralized UPS,' says Google server designer Ben Jai. 'Therefore no wasted capacity.' Efficiency is a major financial factor. Large UPSs can reach 92 to 95 percent efficiency, meaning that a large amount of power is squandered. The server-mounted batteries do better, Jai said: 'We were able to measure our actual usage to greater than 99.9 percent efficiency.' Google has patents on the built-in battery design, 'but I think we'd be willing to license them to vendors,' says Urs Hoelzle, Google's vice president of operations. Google has an obsessive focus on energy efficiency. 'Early on, there was an emphasis on the dollar per (search) query,' says Hoelzle. 'We were forced to focus. Revenue per query is very low.'"
Patents & Catch-22 (Score:5, Informative)
Ah, the catch--22 of the patent--being forced to reveal your hand in order to protect it while underpaid workers at Baidu figure out how to integrate your ideas into their hardware.
Onboard UPS not new (Score:5, Informative)
The in-computer onboard UPS is not a new idea. I don't see how they could have gotten any patents on it since I used it have one of these (my day might still). The device I saw had a gel cell mounted on an 8-bit ISA card, full length. It had +5/12v pass through connectors for powering the drives and it powered the computer through the main bus. There was more logic to it, as it had some monitoring capabilities too.
What's next, patenting a hard drive on a plugin board? Been there, it was called the Hard Card and put a 20mb HDD in an 8 bit full length ISA slot, a truly neat idea for upgrading old XT computers back in the day. You could make them work with AT computers too by putting a regular disk controller, without a drive connected, on the bus too and the BIOS would see the XT controller and boot from it.
Re:The New Mainframe (Score:4, Informative)
By some measurements they exceed the computing power of a mainframe, by others they don't.
Re:Patents & Catch-22 (Score:5, Informative)
considering some of the mini's I worked on had similar setups in additions to external UPS.
then again, we achieve all sorts of power, cooling, and reliability, when we consolidated many "pc" style servers into minis which do the same work. (the heat change alone was staggering)
Oh, for God's sake people... (Score:4, Informative)
Re:Patents & Catch-22 (Score:5, Informative)
Ah, the catch--22 of the patent--being forced to reveal your hand in order to protect it while underpaid workers at Baidu figure out how to integrate your ideas into their hardware.
That's not a catch-22, that's the point. In exchange for everyone learning from what you've done, you get society's protection for a limited number of years.
Also, the workers at Baidu are not underpaid- if they where, they'd leave for better oppurtunities. The workers in question have obviously decided they're better off making stuff for google- they don't need your 'superior' judgement to tell them they should go back to subsistenance farming or melting hazardous materials for precious metals in their homes.
A decision to work, or not to work, and to hire, or not to hire, are based on realistic alternatives, not what some westerner sitting at a keyboard 9,000 miles away thinks is best.
mass servers = "21st century energy refineries"? (Score:3, Informative)
Re:Hey google, want to save some money? (Score:4, Informative)
A desert does not describe the temperature of a region but the (lack of) rainfall/moisture.
http://desertgardens.suite101.com/article.cfm/definition_of_a_desert [suite101.com] (link found using Google).
And besides, put the containers underground and I'm pretty sure that "hot" you refer to becomes a non-issue as well.
Re:No way (Score:4, Informative)
Video: Inside the Container Data Center (Score:5, Informative)
Re:Onboard UPS not new (Score:4, Informative)
Agreed, the onboard UPS is not new. I have a ~10 year old (I believe the CPU is a K6-233) device meant as a SOHO file/print/webserver from IBM that has a built-in gel-cell battery for UPS power just like this server does. Google is 5+ years too late.
Anyone want my prior art to invalidate the patent?
Re:Onboard UPS not new (Score:5, Informative)
I agree!
http://www.globtek.com/html/ups.html [globtek.com]
Re:A quick peek at the picutres says a lot (Score:4, Informative)
Actually, looking at the battery, ir looks like the same exact type of battery as you'd find in an APC small (450-800VA) UPS. We also used the same batteries for emergency power in our door access systems to power the controller when I was managing those at a small college. That type of battery is widely used to compensate for short term power outages.
I presume, given the amount of hardware shown (2 drives, 2 processors, motherboard, RAM) that the battery would probably last that given system about 7-10 minutes... plenty of time for the electric system to failover to the generator farm (you know they have more than 2 for redundancy.
As to the lifetime on those batteries... I was replacing them every 3-3.5 years, maybe 4 if I was lucky. It's a standard generic battery, and the failure rate on them is quite low.
I'd echo another user... If Google wanted to be smart, they wouldn't bother repairing a server when a component fails. Server obselescence at a company that can afford it is about 3-4 years... pretty close to the time for these batteries. They'd probably just pull the main power on it, and when a threshold of servers is "dead" in the container, they pull it offline for renovation... Either to repair the bad servers, or just retire everything.
Re:FCC? UL? (Score:4, Informative)
Re:Patents & Catch-22 (Score:5, Informative)
Please see the Patent Cooperation Treaty [wikipedia.org] which covers this situation; China acceded in 1993, India in 1998. [uspto.gov]
Re:Pretty cool stuff (Score:4, Informative)
then you need to move your offices to the middle of a desert. Space problem solved :)
SMEs often get themselves a small server room, and don't plan for expansion. When the time comes to stick more servers in, they usually have to put them in an office instead, with non-redundant power, little cooling. You're not alone there, but it doesn't necessarily apply to datacentres.
Space at datacentres is often the least of their worries nowadays, (it used to be different), but power is the big problem. Even the DCs in the middle of the metropolis has enough space to fit a few servers, but they can't get the power to them if they did.
Re:99.9% efficiency (Score:4, Informative)
The article says that they use special motherboards that require 12V only, which is what the batteries put out. No conversion needed.
Re:Hey google, want to save some money? (Score:3, Informative)
Plus you also have to account for gradual scaling up & geographical distribution. Easy to do with additional low powered servers, difficult to do with giant expensive mainframes.
Re:Outgassing hydrogen? (Score:3, Informative)
You're Doing It Wrong(tm). A sealed cell will only vent hydrogen if overcharged (at the cost of increasingly reduced cell capacity - you're not filling it back up with water!). An intelligent charger will eliminate any routine hydrogen venting, leaving only the occasional bad battery or battery hooked to a broken charger venting. Google is probably OK with that.
Re:Hey google, want to save some money? (Score:3, Informative)
A 12v battery. I never knew DC was more efficient than AC! WOW GOOGLE IS SO COOL!
I'm not sure if you are being sarcastic. A 12v battery in the power supply is more efficient than taking DC -> AC -> DC. That is what a UPS does, each conversion introduces loss. Having the battery in the power supply means there is no conversion so less power loss.
Re:Hey google, want to save some money? (Score:2, Informative)
Dude... UPS. If you're using the battery, you don't *have* AC.
You do know that a UPS (for the kinds of servers Google is powering) puts out AC power, right?
Oh, you didn't?
Re:Oh, for God's sake people... (Score:3, Informative)
Fair point -- I never thought of that. However, there are some other links to this:
Maybe this is legit...
Re:Hey google, want to save some money? (Score:3, Informative)
A switch mode PSU takes AC, converts it to DC, switches it at a high frequency and then filters it back to DC at each rail voltage. They have obviously modded this PSU so that it can take DC directly in at a much lower voltage and still work so the PSU and UPS are combined. I find this neat.
The whole server is redundant.
Re:The New Mainframe (Score:5, Informative)
Re:They are computers, no more advanced than befor (Score:2, Informative)
There are voltage regulators that can drop/boost the voltage to a predetermined voltage and do so with 90+% efficiency. Look for 'buckboost regulator' or 'switching regulator'.
Tom.
Re:Hey google, want to save some money? (Score:4, Informative)
"If you take the price of a mainframe, and compare that to what google can get for the same money using their current solution, their current solution offers at least 10 times as much cpu performance, and much much more aggregate io(Both hard disk and memory) bandwidth."
no it doesn't.
Plus they are cheaper to maintain, require less power per cycle, require less square feet to house.
Yeah, I actually know about these things.
Re:Hey google, want to save some money? (Score:4, Informative)
When I worked for a University, we bought a few of the largest IBM pSeries machines (power4 at the time). These were powerhouse machines 5 years ago. Each one had a dedicated 24" oversized rack cabinet, and then we had a couple racks just for disk. The 4 machines, and about 40T of Fibre channel disk (or was it DASD), I think it was a total of 128 core and 256GB of ram. I think we paid about a million for that setup.
As was mentioned elsewhere on the webs, the machine shown off by Google was based on Nocona CPUs.. those are atleast 4 years old now. Not likely what they're buying new now.
I bet you could get a base z10 for a few hundred thousand, but a fully loaded one? With a disk array of 750 drives? I bet 4 racks of disk from IBM would cost most of that 950k budget.
Re:Not too difficult, but impressive nonetheless (Score:3, Informative)
Found the patent:
Application number: 11/756,939
Publication number: US 2008/0030078 A1
Filing date: Jun 1, 2007
Inventors: William Whitted, Montgomery Sykora, Ken Krieger, Benchiao Jai, William Hamburgen, Jimmy Clidaras, Donald L. Beaty, Gerald Aigner
Assignee: Exaflop LLC [Note that Exaflop LLC's mailing address is the same as Google's.]
U.S. Classification
307066000
The most interesting parts, to me:
Figure 1, which shows an AC-DC converter, a battery, and a motherboard on a tray, in parallel.
And the following excerpts:
[0013] The system can further include a charger configured to charge the battery through a path connected across the DC bus. In some implementations, the single DC bus voltage is less than about 26 Volts. In some implementations, the single DC bus voltage is between about 10 Volts and about 15 Volts. In some implementations, the single DC bus voltage is about 13.65 Volts. In some implementations, the AC-to-DC conversion circuit regulates the DC output voltage signal to approximately 1 Volt above the maximum nominal charge voltage of the battery. The DC bus voltage can provide sufficient voltage for a linear regulator connected in series with the battery across the DC bus to trickle charge the battery to a fully charged state according to battery specifications.[...]
and this gem:
[0016] The system can further include at least one DC-DC converter configured to convert a voltage supplied on the DC bus to at least one additional DC voltage. In some implementations, the additional voltage is selected from the group consisting of: -5; 1; 3; 3.3; 5; 7.5; 10; about 18-20; and, about 20-26 Volts.
That last one, 16, is pretty specific: It basically comes out and says that there is no secondary regulation to 12V.
And so, I rest my case and declare that it is, indeed, a brilliant and simple design.
Re:The New Mainframe (Score:3, Informative)
Actually google does everything thrice (not unlike the Ramans). And returns the result that reaches it first. So in effect it is even more fault tolerant than the Mainframe. And it does them at different locations not on a single Facility (as opposed to a server or a 1AAA sized Container).
You are underestimating Google.