Ooooh…. liquid cooling video from sC18 https://twitter.com/Yuryu/status/1062178413270786048
On Fri, 9 Nov 2018 at 03:42, Lux, Jim (337K) via Beowulf < beowulf@beowulf.org> wrote: > I’ll bet the surface is rough enough that there are plenty of nucleation > centers. Consider things like leads on parts. > > > > *From: *Beowulf <beowulf-boun...@beowulf.org> on behalf of " > beowulf@beowulf.org" <beowulf@beowulf.org> > *Reply-To: *Prentice Bisbal <pbis...@pppl.gov> > *Date: *Thursday, November 8, 2018 at 7:47 AM > *To: *"beowulf@beowulf.org" <beowulf@beowulf.org> > *Subject: *Re: [Beowulf] More about those underwater data centers > > > > One comment - my dissertation below is specifically about non-ebullient > immersion cooling. As Jim Lux pointed out in a later e-mail, in ebullient > cooling, some kind of surface feature to promote nucleation could be > beneficial. Ebbulient cooling is a whole different beast from normal > (non-ebullient) immersive cooling, since in that case you have changes of > state and gas bubbles flowing through a liquid. > > However, in all of the live and video demonstrations I've seen of Novec, > the processors were completely bare, bubbles were forming at a pretty rapid > rate, so again I think creating some sort of heat sink for this would add > cost with no significant benefit. > > Prentice Bisbal > > Lead Software Engineer > > Princeton Plasma Physics Laboratory > > http://www.pppl.gov > > On 11/08/2018 10:40 AM, Prentice Bisbal wrote: > > Heat fins are used to increase the surface area used for heat transfer, > since the rate of energy transfer by conduction is directly proportional > the surface area. Heat fins are needed when air is involved because air has > such a low thermal conductivity. > > Thermal conductivity of liquids are much high, so heat fins aren't as > necessary. For example, I've read that water can transfer heat orders of > magnitude better than air, so using water to remove hear from a processor > would need orders of magnitude less surface area for the same energy > transfer rate. > > Also, liquids have higher viscosities than gases, so we have to worry > about 'boundary layers'. A boundary layer is area where the edge flowing > fluid is in contact with a solid. The friction between the liquid and the > solid slows down the fluid near the solid. This affects both gases and > liquids, but since liquids have higher viscosities, the effect is more > noticeable. > > Think about a car's radiator - the air side has all the fins on it, and > the liquid side has smooth pipe walls. > > https://en.wikipedia.org/wiki/Boundary_layer > > Convection heat transfer is an equally important mode of heat transfer in > fluids, and in the boundary layer, where the liquids aren't moving as fast, > heat transfer isn't as good, so you need to keep your boundary layer from > becoming too thick. > > Since fluids have much higher thermal conductivities, and boundary layer > effects are more of a concern, I actually think a smooth heat transfer > surface would be better in these immersion cooling cases. I'm sure > smaller, more spaced out fins would probably help heat transfer without > creating too much of a boundary layer, but making those heat sinks adds > cost for increased performance in a situation where it probably isn't > needed. > > Now direct-contact cooling systems like Asetek products do have fins on > the liquid side, if I remember correctly, but that in those systems, there > are pumps to provide forced convection. In immersion cooling, you are > relying on natural convection, so there isn't as much driving force to > overcome viscosity/boundary layer effects to force the liquid through the > heat fins. > > That's my thoughts, anyway. > > Prentice > > On 11/07/2018 04:12 AM, John Hearns via Beowulf wrote: > > Thinking about liquid cooling , and the ebuillient cooling, the main > sources of heat on our current architecture servers are the CPU package and > the voltage regulators. Then the DIMMs. > > Concentrating on the CPU die package, it is engineered with a flat metal > surface which is intended to have a thermal paste to transfer heat across > to a flat metal heatsink. > > Those heatsinks are finned to have air blown across them to transport the > heat away. > > > > In liquid immersion should we be looking at having a spiky surface on the > CPU die packages and the voltage regulators? > > Maybe we should spray the entire board with a 'flocking'' compound and > give it a matt finish! > > I am being semi-serious. I guess a lot of CFD simulation done regarding > air cooling with fins. > > How much work has gone into pointy surfaces on the die package, which > would increase contact area of course and also act as nucleation points for > bubbles? > > > > One interesting experiment to do - assuming the flat areas of the CPU in > an immersive system do not have (non thermal paste) heatsinks bolted on: > > take two systems and roughen up the die package surfacewith sandpaper on > one. Compare temperatures. > > > > ps. I can't resist adding this. Sorry Stu . > https://www.youtube.com/watch?v=kHnifVTSFEo > > I guess Kenneth Williams is a typical vendor Site Engineer. > > pps. the actress in the redress had her career ruined by this film - she > ver got a serious role again after perfectly being typecast. > > > > > > > > > > > > > > > > On Tue, 6 Nov 2018 at 22:57, Prentice Bisbal via Beowulf < > beowulf@beowulf.org> wrote: > > On 11/06/2018 02:03 PM, Lux, Jim (337K) wrote: > > True enough. > > Ebullient cooling does have some challenges – you can form vapor films, > which are good insulators, but if you get the system working right, nothing > beats phase changes for a heat transfer. > > If I recall what I learned in my Transport Phenomena classes in > engineering school, you need a reasonably high temperature difference to > get a stable film like that. For that to happen, radiant heat transfer > needs to be the dominant heat transfer mechanism, in the range of operation > we are talking about, the temperature difference isn't that great, and > conduction is still the dominant form of heat transfer. > > Here's an example of what 3M Novec ebullient cooling looks like. It > doesn't look like it's anywhere near the film boiling regime: > > https://www.youtube.com/watch?v=CIbnl3Pj15w > > -- > Prentice > > > > > > > *From:* Beowulf [mailto:beowulf-boun...@beowulf.org > <beowulf-boun...@beowulf.org>] *On Behalf Of *Prentice Bisbal via Beowulf > *Sent:* Tuesday, November 06, 2018 8:17 AM > *To:* beowulf@beowulf.org > *Subject:* Re: [Beowulf] More about those underwater data centers > > > > . And serviceability is challenging. You need to pull the "wet" boards > out, or you need to connect and disconnect fluid connectors, etc. If > you're in an environment where you can manage that (or are forced into it > by necessity), then you can do it. > > I think everyone on this list already knows I'm no fan of mineral oil > immersion (It just seems to messy to me. Sorry, Stu), but immersion cooling > with other liquids, such as 3M Novec engineered fluid addresses a lot of > your concerns. It as a low boiling point, not much above room temperature, > and it was originally meant to be an electronic parts cleaner (according to > a 3M rep at the 3M booth at SC a few years ago, so if you pull a component > out of it, it dries very quickly and should be immaculately clean. > > The low boiling point is an excellent feature for heat transfer, too, > since it boils from the heat of the processor (ebullient cooling). This > change of state absorbs a lot of energy, making it very effective at > transferring heat away from the processor. The vapor can then rise and > condense on a heat exchanger with a chilled water heat exchanger, where it > again transfers a lot of heat through a change of state. > > Prentice > > On 11/05/2018 06:30 PM, Stu Midgley wrote: > > I refute both these claims. > > > > You DO want to run your boards immersed in coolant. It works wonderfully > well, is easy to live with, servicing is easy... and saves you almost 1/2 > your power bill. > > > > People are scared of immersion cooling, but it isn't that difficult to > live with. Some things are harder but other things are way easier. In > total, it balances out. > > > > Also, given the greater reliability of components you get, you do less > servicing. > > > > If you haven't lived with it, you really have no idea what you are missing. > > > > > > Serviceability is NOT challenging. > > > > > > > > You really do NOT want to run boards immersed in coolant - yeah, there's > folks doing it at HPC scale > > Whatever the coolant, it leaks, it oozes, it gets places you don't want it > to go. And serviceability is challenging. You need to pull the "wet" boards > out, or you need to connect and disconnect fluid connectors, etc. If > you're in an environment where you can manage that (or are forced into it > by necessity), then you can do it. > > -- > > Dr Stuart Midgley > sdm...@gmail.com > > > > _______________________________________________ > > Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing > > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf > > > > > > _______________________________________________ > Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf > > > > > _______________________________________________ > > Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing > > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf > > > > > > _______________________________________________ > Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf >
_______________________________________________ Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
