Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 



Forgot your password?
typodupeerror
×
Databases Data Storage Earth Intel

Will Submerging Computers Make Data Centers More Climate Friendly? (oregonlive.com) 138

20 miles west of Portland, engineers at an Intel lab are dunking expensive racks of servers "in a clear bath" made of motor oil-like petrochemicals, reports the Oregonian, where the servers "give off a greenish glow as they silently labor away on ordinary computing tasks." Intel's submerged computers operate just as they would on a dry server rack because they're not bathing in water, even though it looks just like it. They're soaking in a synthetic oil that doesn't conduct electricity. So the computers don't short out.

They thrive, in fact, because the fluid absorbs the heat from the hardworking computers much better than air does. It's the same reason a hot pan cools off a lot more quickly if you soak it in water than if you leave it on the stove.

As data centers grow increasingly powerful, the computers are generating so much heat that cooling them uses exorbitant amounts of energy. The cooling systems can use as much electricity as the computers themselves. So Intel and other big tech companies are designing liquid cooling systems that could use far less electricity, hoping to lower data centers' energy costs by as much as a third — and reducing the facilities' climate impact. It's a wholesale change in thinking for data centers, which already account for 2% of all the electricity consumption in the U.S... Skeptics caution that it may be difficult or prohibitively expensive to overhaul existing data centers to adapt to liquid cooling. Advocates of the shift, including Intel, say a transition is imperative to accommodate data centers' growing thirst for power. "It's really starting to come to a head as we're hitting the energy crisis and the need for climate action globally," said Jen Huffstetler, Intel's chief product sustainability officer...

Cooler computers can be packed more tightly together in data centers, since they don't need space for airflow. Computer manufacturers can pack chips together more tightly on the motherboard, enabling more computing power in the same space. And liquid cooling could significantly reduce data centers' environmental and economic costs. Conventional data centers' evaporative cooling systems require tremendous volumes of water and huge amounts of electricity...

Many other tech companies are backing immersion cooling, too. Google, Facebook and Microsoft are all helping fund immersion cooling research at Oregon State... [T]he timing may finally be right for data centers operators to make the shift away from air cooling to something far more efficient. Intel's Huffstetler said she expects to see liquid cooling become widespread in the next three to five years.

The article notes other challenges:
  • liquid adds more weight than some buildings' upper floors can support
  • Some metals degrade faster in liquid than they do in air.
  • And the engineers had to modify the servers by removing their fans — "because they serve no purpose while immersed."

This discussion has been archived. No new comments can be posted.

Will Submerging Computers Make Data Centers More Climate Friendly?

Comments Filter:
  • by EmagGeek ( 574360 ) on Monday June 12, 2023 @06:37AM (#63594888) Journal

    Sounds like the answer is "no."

    • Headline should've read, "...make data centres cheaper to run?" That's essentially what this is about.
    • by caseih ( 160668 )

      I agree the answer is no, but the use of oil petrochemicals is not the reason. It's not like the oils are being burned emitting net carbon. Even in a carbon-neutral, zero-fossil-fuel world, oils will still need to be refined, recycled, and used for a variety of industrial and mechanical purposes that don't emit net carbon, such as bearings, gearboxes, and thermal management systems.

      Or did you think Teslas use some sort of magical zero-friction, spherical cow bearings?

      • No, they are not themselves being burned... but they will be, as the recycling stream for them will likely involve re-refining into either motor oil or heavy fuel for ships. Plus there's the environmental risk of leaks, spillage, etc etc..

        Overall it is not worth the cost or the risk. But, it will enable the virtue-signaling, so by all means it must be all ahead flank.

        • Overall it is not worth the cost or the risk.

          Until you do some type of analysis comparing air-cooled vs. oil-cooled, you don't actually know if this is true or not. If these oils last for years or decades, it could easily make up for their environmental impact by using less energy.

          • Oil is a biological hydrocarbon. You can turn it into anything. Hell, there is bacteria that digests it. Sometimes after an oil spill you get a plankton bloom. You can also burn it and run the CO2 into scrubbers, or plant trees to offset what you burn. But it is a hella useful liquid. From hydraulics to lubrication, heat conductivity, etc. Oil or water cooled computers would spend less energy moving air and cooling air. It is also easier to maintain water temp in a closed system than air. So, yeah, it is pr
            • by caseih ( 160668 )

              Correct. And regardless of EV vehicles and net-zero emissions, we will continue to need some oil indefinitely for many purposes, including cooling like this, and especially lubrication of anything mechanical. Just not for burning for energy.

              But if oils can be produced in a carbon-neutral way, they have far more energy storage than any battery. Nothing wrong with combustion if the carbon source is neutral.

  • Huh? (Score:4, Insightful)

    by Ol Olsoc ( 1175323 ) on Monday June 12, 2023 @06:40AM (#63594894)
    I didn't pay a tremendous amount of attention since it isn't my bailiwick, but I recall submergible servers more than a decade ago.

    Noting that the proposed liquid is derived from petrochemicals, I'm not certain how "climate friendly" that is.

    • IBM did this in the 90s already so this is hardly anything new, only the scale they're talking about is different.

      It comes up every few years in mainstream media.

      • Re:Huh? (Score:5, Informative)

        by LordHighExecutioner ( 4245243 ) on Monday June 12, 2023 @08:06AM (#63595096)
        Seymour Cray did this before, with his Cray-2 [wikipedia.org], that was cooled in a Fluorinert [wikipedia.org] bath.
        • by dargaud ( 518470 )
          IIRC it wasn't a bath but a spray directly on the CPU, where it would evaporate to be condensed somewhere else.
          • IIRC it wasn't a bath but a spray directly on the CPU, where it would evaporate to be condensed somewhere else.

            Uh, no. The entire system was submerged in a 'fishbowl' of sorts and you had to drain the thing to replace a module. There was plumbing and pumps to drain and refill it pretty quickly. I worked for Cray as a software guy until SGI bought us out; I worked on many of those systems. Not that we sold very many...

            • by dargaud ( 518470 )
              I stand corrected ! This kind of stuff was mythical at the time. We had a T3D and another model in one of our labs, but I wasn't allowed to approach either at the time !!! My favorite Seymour Cray quotes:

              "If you were plowing a field, which would you rather use? Two strong oxen or 1024 chickens ?" - about clusters.

              "Anyone can build a fast CPU. The trick is to build a fast system."

              "When he was told that Apple Computer had just bought a Cray to help design the next Apple Macintosh, Seymour Cray commented that he had just bought a Macintosh to design the next Cray."

              "Memory is like an orgasm. It's a lot better if you don't have to fake it." - on virtual memory.

    • Re:Huh? (Score:4, Informative)

      by Nutria ( 679911 ) on Monday June 12, 2023 @07:49AM (#63595044)

      They aren't *burning* the oil. That is the climate-unfriendly part of petroleum...

      • They aren't *burning* the oil. That is the climate-unfriendly part of petroleum...

        But they are extracting and then cracking the crude oil they use.

        • by Nutria ( 679911 )

          So?

          • So?

            Compare extracting, then cracking oil, to plain old air.

            • by Shaitan ( 22585 )

              Well they are already extracting and cracking oil. This is almost certainly using a byproduct like mineral oil, in fact, it probably is mineral oil.

            • The correct comparison is between the oil that would recirculate indefinitely vs. the oil/coal that would be burned to create electricity to cool the air that cools the datacenter.

              That the oil is a petrochemical is really a silly detail to pick up on. It's a misdirection, like fretting about the eventual disposal of wind turbine blades. Using petrochemicals for indefinite reuse is just not the same as one-time or open-loop consumption where you can see an entire tanker truck stopping by to unload literal

      • Re:Huh? (Score:5, Insightful)

        by Freischutz ( 4776131 ) on Monday June 12, 2023 @08:07AM (#63595100)

        They aren't *burning* the oil. That is the climate-unfriendly part of petroleum...

        No, petrochemicals are also pretty toxic and extracting the petroleum is also a very climate-unfriendly process. Case in point: https://news.slashdot.org/stor... [slashdot.org]

        • It's all relative. The production of petrochemicals isn't green, but it pales in comparison to the environmental impact of burning them or disposing of them incorrectly.

          That Slashdot article you linked references bad actors (actually it references the worst actor), not a fundamental industrial process shortcoming. It's like saying humans are fundamentally pieces of **** and pointing to Hitler's Wikipedia entry for proof.

          • It's all relative. The production of petrochemicals isn't green, but it pales in comparison to the environmental impact of burning them or disposing of them incorrectly.

            That Slashdot article you linked references bad actors (actually it references the worst actor), not a fundamental industrial process shortcoming. It's like saying humans are fundamentally pieces of **** and pointing to Hitler's Wikipedia entry for proof.

            Problem is most actors are bad actors because that's where the profits are. The more research is done into this area the more of these mega emitters are discovered.

      • They aren't *burning* the oil. That is the climate-unfriendly part of petroleum...

        Except for the methane emissions - methane is a much more potent greenhouse gas than CO2 - and the energy used in extraction. Not to mention transport, evaporation of unhealthy VOC's, energy used in refining, and oil spills. Probably others as well.

    • There are companies that already do this. Dug have an oil-immersion supercomputer in Texas: https://dug.com/ [dug.com]

      The liquid they use is more like baby-oil than anything else. And they don't burn it so it's pretty climate friendly.
      • Re: (Score:2, Funny)

        by Anonymous Coward
        I didn't know you could get oil from babies. Poo, barf, sure...
    • by Shaitan ( 22585 )

      Perfectly climate friendly since they aren't burning the oils. It sounds suspiciously like mineral oil.

      • Perfectly climate friendly since they aren't burning the oils. It sounds suspiciously like mineral oil.

        I think you are right about the mineral oil. But let's remember, it has to be refined, which does take energy.

        I'd hate to be the technician that has to deal with the oil soaked servers. Especially after they are hot. My guess is the entire server gets chucked for any issues at all, and sent to the landfill. Unless they degrease the things, which adds it's own environmental/safety issues.

        This isn't new technology, but it is surely messy technology.

        • by Shaitan ( 22585 )

          "This isn't new technology, but it is surely messy technology."

          No doubt about that.

          "But let's remember, it has to be refined, which does take energy."

          I thought it was a commercialized byproduct like petroleum jelly?

          Of course cooling with air isn't an even remotely energy free process. Look at how much more efficient those multi-zone A/C systems are than the old school central air systems, all because they get rid of the air ducting and just move the heat via fluid. Similarly I imagine they could run a loop

        • I'd hate to be the technician that has to deal with the oil soaked servers.

          People have to deal with oil/grease/etc. in equipment all the time. It's not a big issue.

          My guess is the entire server gets chucked for any issues at all, and sent to the landfill.

          I HOPE you don't think people throw out equipment just because it has some oil/grease on it.

    • by caseih ( 160668 )

      What it is exactly about petrochemicals that is inherently "climate [unfriendly]?" Are you mistaking the every-day use of oils with burning them? Do you really think electric cars need no grease in the bearings, or oil to bathe gearboxes?

      I know it's intellectually easy to just say "oil bad" but the truth is far more nuanced and nearly everything relies on oil in some small way that is not contributing to global warming, not releasing carbon, and not being burned. And mostly recycled actually.

      And it's als

      • What it is exactly about petrochemicals that is inherently "climate [unfriendly]?" Are you mistaking the every-day use of oils with burning them?

        Explain how the refinement of crude oil into it's constituents is made.

        Do you really think electric cars need no grease in the bearings, or oil to bathe gearboxes?

        Do you really think that I ever said such a thing? I don't think I did. Although I will be quick to apologize once you point out exactly where I saidthat since you think I said that

        I know it's intellectually easy to just say "oil bad"

        Not even a tenth as intellectually lazy as insinuating that I said "Oil bad". Come on homie - let's stick to the conversation, this is not some soundbite news thing with all the rhetorical tricks here.

        I am going to put it a different way now.

        We have t

  • Because while the end-stage of the cooling is both simpler and harder (cannot have some hardware submerged), the target temperature and amount of heat to remove stays the same? All you get is a possible space advantage over airflow-cooling and smaller heatsinks. That does not do much for climate-change.

    • Re: Ah, no? (Score:4, Interesting)

      by Dantu ( 840928 ) on Monday June 12, 2023 @07:04AM (#63594942)
      Energy required does not scale with heat moved, but rather with heat*(deta temperature). Since air has a much lower heat capacity than oil, they have to move a LOT more of it, or cool it more, or both. Those take extra power. In particular, consider a server farm in a relatively cool area, like Virginia. The chips themselves might be operating at 40-60c, a higher temperature than the air outside. With perfect heat transfer you would need no energy to cool them, in theory you could even reclaim some power. In practice transferring that heat outside is difficult and it may actually be more efficient to run an air conditioner than to try to push that much ambient/passively heat exchanged air through your system.
      • by Luthair ( 847766 )
        Sure, but oil being denser than air also requires more energy to circulate. I believe the poster is correct, without infinite volume one still needs a way to cool the oil. Its obvious that liquids can absorb more heat than a gas, but as a computer scientist not an engineer the relative efficiencies of the two systems (pump vs fan, etc.) isn't clear.
        • by torkus ( 1133985 )

          You're already pumping chillwater to your CRAH's so that more or less net's out. You're just skipping the 100's of fans everywhere from air handlers to server chassis.

    • > That does not do much for climate-change.

      No indeed. However, there'd be less volume of liquid to cool than air, so the practicalities of it might be easier. Seems doubtful to me, but who knows...?

      I'd also worry this will lead to more abuse of the "infinite ocean" problem. That is, that a lake might look like an enormous heat sink, and the datacentre might only change its temperature by 1% and, and, and... Yet the lake will heat up more than it should, and that'll affect its wildlife and other factors i

      • by Zak3056 ( 69287 )

        if you're going submerge your servers in something, why not make it a lake?

        I would expect that replacing the water in the lake with mineral oil would likely have a deleterious effect on the wildlife you were so worried about in the paragraph before this one.

    • Because while the end-stage of the cooling is both simpler and harder (cannot have some hardware submerged), the target temperature and amount of heat to remove stays the same? All you get is a possible space advantage over airflow-cooling and smaller heatsinks. That does not do much for climate-change.

      Yes, but your conclusion is incomplete. With liquid based cooling (of which water cooling with a piped setup would also qualify), you need to move a lot less liquid, and the liquid can be hotter for a target

    • All you get is a possible space advantage over airflow-cooling and smaller heatsinks.

      An oil bath may well be more efficient. I'm not a fluid mechanics specialist so I may be off base, but I imagine airflow creates more energy-wasting vortices than oil flow, by allowing some hot air to 'circle back' to the source

      Also, running the hot oil through insulated pipes to a heat exchanger might result in better, more efficient use of the waste heat. Just a guess though.

    • by torkus ( 1133985 )

      Server fans draw more power than most people realize. A quick google puts that number at 10-15% of the server power.

      You can easily spend a couple kW just moving air within a rack. While the cooling liquid is not as thin as air, it's more like water than motor oil. The CRAH units still need to pump chillwater and run huge fans to blow air...etc. so it's not like we aren't pumping around liquids already. The savings come from removing 100's of fans. Even the 1" redundant little server fans can draw dozen

  • by chas.williams ( 6256556 ) on Monday June 12, 2023 @07:07AM (#63594950)
    We used flourinert instead of mineral oil and tried using COTS equipment. We had everything submerged, including power supplies. During operation, tin whiskers would grow on the high-voltage parts and eventually short. Any switching regulators also seemed to experience this to some degree. At that point, we started to conformally coat the boards to avoid this issue. Any wiring needed Teflon jacketing, because flourinert would remove plasticizers creating a brittle mess. However, you could get some fantastic energy savings for cooling because you just needed to condense the flourinert and even an 85F day could do that without compressors.
    • by Ol Olsoc ( 1175323 ) on Monday June 12, 2023 @07:20AM (#63594984)

      We used flourinert instead of mineral oil and tried using COTS equipment. We had everything submerged, including power supplies. During operation, tin whiskers would grow on the high-voltage parts and eventually short. Any switching regulators also seemed to experience this to some degree. At that point, we started to conformally coat the boards to avoid this issue. Any wiring needed Teflon jacketing, because flourinert would remove plasticizers creating a brittle mess. However, you could get some fantastic energy savings for cooling because you just needed to condense the flourinert and even an 85F day could do that without compressors.

      I didn't know about this foray, but yeah, that's a show stopper. Tin whiskers are the salty bitch of electronics, they've even knocked out satellites https://nepp.nasa.gov/WHISKER/... [nasa.gov] They are why you can still get tin/lead solder. The alternatives don't do too well.

      Where your servers using lead free solder?

      • by tlhIngan ( 30335 )

        I didn't know about this foray, but yeah, that's a show stopper. Tin whiskers are the salty bitch of electronics, they've even knocked out satellites https://nepp.nasa.gov/WHISKER/ [nasa.gov]... They are why you can still get tin/lead solder. The alternatives don't do too well.

        It turns out that tin whiskers don't come from solder, but other tin-plated items used.

        Tin is a strange material - the whiskers don't seem to be drawing from areas nearby - the material that forms the whiskers seems to be coming from the bulk.

    • by Afell001 ( 961697 ) on Monday June 12, 2023 @07:26AM (#63594996)
      Fluorinert is a chemical-cousin to the CFCs used as refrigerants, so it makes sense that you could just condense any of the Fluorinert that sublimates to a gas. And dealt with the brittle wire mess - learned that lesson hard. Anything exposed needed silicone grease conformal coating. Also did mineral oil baths - entirely different problems. Wicking and capillary action was real bad - everything eventually has a thin coat of oil. And bugs. I can't explain why they like to climb into the oil - it is certain death, like mosquitos caught in amber. So you have to drain and filter it all eventually.
    • During operation, tin whiskers would grow on the high-voltage parts and eventually short.

      Any idea why that's worse with fluorinert vs air?

      • Surprisingly we know very little about tin whiskers. We don't fully understand how or why they grow in air, much less other materials.

    • I think a lot of this problem comes from the fact that the components weren't made with the intention that they could be submerged. It will require research into finding suitably inexpensive replacements that function in a different environment but there is no reason to believe it cannot be done. I think that the server industry is large enough that it could sustain the production of different chemistries for PCBs and wiring jackets.

      • by Junta ( 36770 )

        The question is whether it is worth it, even for the server industry.

        Even if you each socket needs 1.5U of space to be effectively air cooled, you are still at better density than where you can go with immersion cooling. Sure you can have them closer together, but it's going to be shorter. Alternatively you could have assemblies where you pull out multiple boards depth wise, but that's going to be a massive frustration to manage, even moreso than immersion cooling in general.

        It isn't as "sexy" as immersio

    • So what was the result? It sounds like it was an experiment that was not successful - mitigation cost > savings?

  • "motor oil-like petrochemicals"

    Does not sound environmentally friendly to me. Where do you dispose of all of that when you are finished with it?

  • They mention PFAS. Is the Fluorinert used in some Cray machines one of them? I remember it being quite expensive, but would economy of scale fix that if demand were orders of magnitude higher than in the Cray days?

  • It's the same reason a hot pan cools off a lot more quickly if you soak it in water than if you leave it on the stove.

    Oh, so THAT'S WHY it happens! I always thought the hot pan was just getting distracted by taking a bath.

  • Submersion cooling is nothing new. Other commenters here even pointed out CRAY's use of liquid coolant. Let me state for the record that, if this worked well, we would have it already. And to those that disagree: what new technology has come about in the recent resurgence that we didn't have back in the 1980s?

    Adding a liquid brings so many complications: liquid coolant contamination, serviceability, handling (enclosures). Just imagine being the guy in the data center tasked with changing hard drives!

    Perha

    • by Junta ( 36770 )

      Indeed, nothing new, we've long had there contenders for cooling: air, plumbed liquid, and liquid immersion.

      Historically, it's no contest: air cooling. Many inefficiencies stem from largely using 40mm fans with tiny fins when you go for density, but even then that's not *too* bad. A great deal of efficiency is recouped by chassis that share at *least* 80mm fans. When it came to density, plumbed liquid and air would tie (non-cooling considerations limited density to be comfortably cooled by air), and imme

    • by xvan ( 2935999 )
      Solid state disks are new. Now you have computers with no moving parts besides fans, and transformer vibrations, but those can be isolated using power rails.
  • TL;DR - energy in is energy out. Submersion in oil, water, mineral spirits, or refrigeratnt only matters to MOVING that energy, not to its creation nor destruction.
    Hint: you don't destroy energy. See the second law of thermal dynamics.

    Conclusions on climate forgiveness are at the last paragraph. If you care about the climate, skip to that.

    ---
    Big pic:

    You have a device, let's call it a "server" but it could be a car engine too. When activated it takes in energy. If it helps that's 60W for a 60W incandesc

    • by Junta ( 36770 )

      You are right that the best thing is overall efficiency. I'll also add that computing actually tends to have an efficiency peak before the 'top end performance'. So if you clock down a few, you end up with maximum work per unit energy, at the expense of work per time.

      However, air cooling can be done more efficiently if you first move the heat to a more amenable context.

      In a datacenter, you are almost certainly using air to extract heat, and then extracting the heat from the air using an air-water heat exc

  • Have the issues with connectors been resolved in current iterations of this technology? 20 years ago that was the challenge and the last I had heard there were no meaningful improvements.

  • Not sure about other parts of the world but I know the shallow mines I have visited generally still had snow and ice in the middle of summer.
  • Not new, also... (Score:4, Informative)

    by bradley13 ( 1118935 ) on Monday June 12, 2023 @07:46AM (#63595034) Homepage

    ...it's called "mineral oil". Journalists really shouldn't write about things they don't understand.

    This might save a bit of energy in terms of fans, but it also complicates maintenance. If there is a real advantage, it is likely to be density - packing more servers into a rack.

  • Not this again... (Score:5, Interesting)

    by ctilsie242 ( 4841247 ) on Monday June 12, 2023 @07:59AM (#63595074)

    I have had to deal with some high-zoot vendor appliances that were immersion cooled:

    1: They are messy. Replace a board requires gloves and a lot of additional work (hope your glasses don't plop in the vat). Think replacing that hard disk or PSU is bad on that server, wait until you have to deal with liquid everywhere. It also tended to leak, and had to have a drip pan with a sump pump to have the spilled coolant go into a container to be disposed of properly.

    2: The appliance wound up just having the main heat exchanger be in the air, so all the CRACs could cool it without needing multiple liquid heat exchangers.

    3: The coolant wasn't exactly environmentally friendly.

    4: If you looked at the setup wrong, it would leak. Think a custom, open-loop PC except worse. Especially if a thermal cycle hits, like the data center losing CRACs or a power loss. Yes, in theory, this should never happen... but ice storms and vehicles hitting power lines are a thing. Plus, it didn't use easily replaceable fittings when one started leaking. One had to have a vendor tech come out with custom sized plumbing, and that could take days to order what was needed.

    5: The coolant was only available from the manufacturer, and was sold at crazy prices/liter.

    Overall, I'd find another system to get away from that. Maybe get governments to work using solid coolants, as in electromagnetic refrigeration, just so that this form of liquid cooling can be killed off. Closed loop cooling isn't bad, but even with that, there is a good chance those might leak and ruin board components. Wish there were a way to make Peltier cooling more efficient.

    Immersion cooling might look interesting, but has a ton of downsides. Best thing to do is set it aside and focus on something like magnetic cooling or some way to cool on a solid level so a liquid coolant is not needed. Until then, better work on heat pipes and exchangers is probably our best bet.

    • Immersion cooling might look interesting, but has a ton of downsides.

      Right now you've mainly listed some operational complaints. These are historically minor downsides compared to ongoing cost and inefficiencies of air cooling. Your number 2 complaint (that in the end its still an air-cooled CRAC) seemingly confirms that you don't understand the problem being addressed here, otherwise you wouldn't have mentioned it as a downside.

      Best thing to do is set it aside and focus on something like magnetic cooling

      No, the best thing to do is not throw your eggs in one R&D basket and hope for magical breakthroughs. It may shock you but a company which emplo

  • Long term, why not invest in things like taking atomic-level data storage from something done in a laboratory, to something practical for server use?
    Going from the equivalent of a million atoms to represent a bit of data, down to 9 atoms, or even a 1:1 (which has been done in laboratory settings if I recall correctly) would be... well, it'd be mind blowing just how much (at least on a server / data center storage level) things would shrink - including the size of the facility itself, and the amount of ene
  • This might be a silly question, but...

    The same amount of heat is still initially generated, the liquid just does a better (faster) job of pulling the heat away from the computers. But the heat is still there, contained within the liquid. You still need to remove the heat from the liquid, so where does it go and what is the mechanism to draw it out?

    "The clear fluid looks just like water, flowing gently through the tank that holds the servers, pumped up from underneath and then spilling over the top like an i

    • They probably run it through some sort of radiator with forced air.

    • Don't we end up going back to air or water-evaporative cooling to cool the cooling liquid?

      Yes we do, but that's not the issue. By having a fluid that is a better conductor of heat you not only can make the interface smaller (more parts in less space), but you can wick off concentrated heat better (there's a reason why modern CPU coolers require heatpipes / vapour chambers despite still ultimately being aircoolers), but it also means you don't need to cool the fluid to as low of a temperature which makes those heat exchangers more efficient to run as efficiency increases with delta-T of the fluid

    • The rate of heat exchange off the board is proportional to the amount of "stuff" that comes in close contact. Liquid is "more stuff" than gas. Mineral oil is ballpark 100X as dense per cubic centimeter as air.

      What that means you can pull 100X times as much heat away from the board with the same volume of material flowing by. Now, such things are not equal, because it is sure easier to blast air through a server than liquid.

      But I do not think these two stages are where the big win comes from, because they

  • See the Cray-2. [hpe.com]

  • I remember seeing lots of people submerging PCs in all sorts of liquids years ago to see what they could do.

    All the liquids they tried had some drawbacks and for a variety of reasons, none of the techniques ever gained any traction.

    I think Linus Tech Tips did something along those lines as did several other well-known tech personalities.

    Whatever you submerge it in has to be chemically neutral, have good heat conductivity, and not be toxic. There aren't many liquids that fit the bill back then; maybe there's

    • by Shaitan ( 22585 )

      That's with off the shelf gear, there are also issues with expansion. But given how much the idea comes up I'm surprised we never really saw gear that was made to be submerged as opposed to just submerging gear that wasn't made for it and hoping it works out as is.

  • WEDOS hosting [wedos.com] started experimenting with servers in oil in 2013 and for some years they are already running servers in oil in production: https://youtu.be/VCQP1pX3FHI?t... [youtu.be]
  • "Intel's submerged computers operate just as they would on a dry server rack because they're not bathing in water, even though it looks just like it. They're soaking in a synthetic oil that doesn't conduct electricity. So the computers don't short out."

    I don't know if intel is aware of this or not but water also doesn't conduct electricity.

    • Comment removed based on user account deletion
      • by Shaitan ( 22585 )

        "PURE Water doesn't conduct electricity. Keeping water pure, is I suspect, the problem."

        That is absolutely the problem. I'm just pointing out TFS is incorrect.

        "~ water has a relatively low boiling point, low viscosity, maybe isn't as effective at absorbing heat, etc."

        Assuming they are just getting fancy and talking about mineral oil the water is better, which is why we have fancy water cooling systems and not oil cooling systems in our consumer gear.

      • by PPH ( 736903 )

        ~ water has a relatively low boiling point, low viscosity, maybe isn't as effective at absorbing heat, etc.

        Water is quite effective at absorbing heat. It has one of the highest specific heats [engineeringtoolbox.com] of common substances. As you can see from this chart, only a few are better. Like hydrogen. But ...
        Oh, the humanity!*

        *Actually, not so much of a problem, as hydrogen is already used for equipment cooling [wikipedia.org].

  • As everybody above said "been there, done that."

    A company tried that with some pcie cards a former company I worked for made. And a demo of one of our helium filled drives immersed in liquid is still running in one of the labs.

  • https://en.wikipedia.org/wiki/... [wikipedia.org]

    Liquid cooling allows higher circuit density, but requires an extra coolant-to-external heat exchanger, along with all of the complexity and messiness of dealing with large amounts of liquid coolant. You can also have unintended chemistry between the electronics and the coolant.

    Not crazy but its been tried for almost half a century and so far as only found a few niche applications where it made sense. It might make sense here, but that's not the way I'd bet
  • But you'll need plumbers.

  • Comment removed based on user account deletion
  • Imagine what the fishtank screensaver folks would think.

  • Sounds like they are going to be generating even more heat, with denser boards and denser racks. They still need to remove all that heat and it all has to go somewhere. Unless I am misunderstanding some basic physics, they still need heat exchangers and chillers, whether they are cyclical compression chillers or evaporative swamp coolers, they somehow have to get the heat out of the servers and into the external environment somehow.
    I can believe better efficiency with full immersion heat transfer, but get

  • I am a pioneer in experimental Cuisine-Assisted Computer System Engineering at the North Dakota Institute of Technology, where I use salad oil to cool computers. One of my thesis candidates has experimentally used ranch dressing to cool Intel CPUs, with promising results. Another introduced vinaigrette for 1U blades. We believe the future is bright for these farm-based environmentally safe technologies.

  • ISTR that like a decade ago, there was a lot of work being done on building hardware that could run at 70C or more, so that cooling was nothing more than bringing fresh air from the outside, blowing it through the room, then exhausting it back outside. One use case for this was small portable modular data centers built inside shipping containers.

  • it's the sunken cost fallacy,

A good supervisor can step on your toes without messing up your shine.

Working...