I’ll bet the surface is rough enough that there are plenty of
nucleation centers. Consider things like leads on parts.
FROM: Beowulf <beowulf-boun...@beowulf.org> on behalf of
"beowulf@beowulf.org" <beowulf@beowulf.org>
REPLY-TO: Prentice Bisbal <pbis...@pppl.gov>
DATE: Thursday, November 8, 2018 at 7:47 AM
TO: "beowulf@beowulf.org" <beowulf@beowulf.org>
SUBJECT: Re: [Beowulf] More about those underwater data centers
One comment - my dissertation below is specifically about
non-ebullient immersion cooling. As Jim Lux pointed out in a later
e-mail, in ebullient cooling, some kind of surface feature to
promote nucleation could be beneficial. Ebbulient cooling is a whole
different beast from normal (non-ebullient) immersive cooling, since
in that case you have changes of state and gas bubbles flowing
through a liquid.
However, in all of the live and video demonstrations I've seen of
Novec, the processors were completely bare, bubbles were forming at
a pretty rapid rate, so again I think creating some sort of heat
sink for this would add cost with no significant benefit.
Prentice Bisbal
Lead Software Engineer
Princeton Plasma Physics Laboratory
http://www.pppl.gov
On 11/08/2018 10:40 AM, Prentice Bisbal wrote:
Heat fins are used to increase the surface area used for heat
transfer, since the rate of energy transfer by conduction is
directly proportional the surface area. Heat fins are needed when
air is involved because air has such a low thermal conductivity.
Thermal conductivity of liquids are much high, so heat fins aren't
as necessary. For example, I've read that water can transfer heat
orders of magnitude better than air, so using water to remove hear
from a processor would need orders of magnitude less surface area
for the same energy transfer rate.
Also, liquids have higher viscosities than gases, so we have to
worry about 'boundary layers'. A boundary layer is area where the
edge flowing fluid is in contact with a solid. The friction between
the liquid and the solid slows down the fluid near the solid. This
affects both gases and liquids, but since liquids have higher
viscosities, the effect is more noticeable.
Think about a car's radiator - the air side has all the fins on it,
and the liquid side has smooth pipe walls.
https://en.wikipedia.org/wiki/Boundary_layer
Convection heat transfer is an equally important mode of heat
transfer in fluids, and in the boundary layer, where the liquids
aren't moving as fast, heat transfer isn't as good, so you need to
keep your boundary layer from becoming too thick.
Since fluids have much higher thermal conductivities, and boundary
layer effects are more of a concern, I actually think a smooth heat
transfer surface would be better in these immersion cooling cases.
I'm sure smaller, more spaced out fins would probably help heat
transfer without creating too much of a boundary layer, but making
those heat sinks adds cost for increased performance in a situation
where it probably isn't needed.
Now direct-contact cooling systems like Asetek products do have
fins on the liquid side, if I remember correctly, but that in those
systems, there are pumps to provide forced convection. In immersion
cooling, you are relying on natural convection, so there isn't as
much driving force to overcome viscosity/boundary layer effects to
force the liquid through the heat fins.
That's my thoughts, anyway.
Prentice
On 11/07/2018 04:12 AM, John Hearns via Beowulf wrote:
Thinking about liquid cooling , and the ebuillient cooling, the main
sources of heat on our current architecture servers are the CPU
package and the voltage regulators. Then the DIMMs.
Concentrating on the CPU die package, it is engineered with a flat
metal surface which is intended to have a thermal paste to transfer
heat across to a flat metal heatsink.
Those heatsinks are finned to have air blown across them to
transport the heat away.
In liquid immersion should we be looking at having a spiky surface
on the CPU die packages and the voltage regulators?
Maybe we should spray the entire board with a 'flocking'' compound
and give it a matt finish!
I am being semi-serious. I guess a lot of CFD simulation done
regarding air cooling with fins.
How much work has gone into pointy surfaces on the die package,
which would increase contact area of course and also act as
nucleation points for bubbles?
One interesting experiment to do - assuming the flat areas of the
CPU in an immersive system do not have (non thermal paste) heatsinks
bolted on:
take two systems and roughen up the die package surfacewith
sandpaper on one. Compare temperatures.
ps. I can't resist adding this. Sorry Stu .
https://www.youtube.com/watch?v=kHnifVTSFEo
I guess Kenneth Williams is a typical vendor Site Engineer.
pps. the actress in the redress had her career ruined by this film -
she ver got a serious role again after perfectly being typecast.
On Tue, 6 Nov 2018 at 22:57, Prentice Bisbal via Beowulf
<beowulf@beowulf.org> wrote:
On 11/06/2018 02:03 PM, Lux, Jim (337K) wrote:
True enough.
Ebullient cooling does have some challenges – you can form vapor
films, which are good insulators, but if you get the system working
right, nothing beats phase changes for a heat transfer.
If I recall what I learned in my Transport Phenomena classes in
engineering school, you need a reasonably high temperature
difference to get a stable film like that. For that to happen,
radiant heat transfer needs to be the dominant heat transfer
mechanism, in the range of operation we are talking about, the
temperature difference isn't that great, and conduction is still the
dominant form of heat transfer.
Here's an example of what 3M Novec ebullient cooling looks like. It
doesn't look like it's anywhere near the film boiling regime:
https://www.youtube.com/watch?v=CIbnl3Pj15w
--
Prentice
FROM: Beowulf [mailto:beowulf-boun...@beowulf.org] ON BEHALF OF
Prentice Bisbal via Beowulf
SENT: Tuesday, November 06, 2018 8:17 AM
TO: beowulf@beowulf.org
SUBJECT: Re: [Beowulf] More about those underwater data centers
. And serviceability is challenging. You need to pull the "wet"
boards out, or you need to connect and disconnect fluid connectors,
etc. If you're in an environment where you can manage that (or are
forced into it by necessity), then you can do it.
I think everyone on this list already knows I'm no fan of mineral
oil immersion (It just seems to messy to me. Sorry, Stu), but
immersion cooling with other liquids, such as 3M Novec engineered
fluid addresses a lot of your concerns. It as a low boiling point,
not much above room temperature, and it was originally meant to be
an electronic parts cleaner (according to a 3M rep at the 3M booth
at SC a few years ago, so if you pull a component out of it, it
dries very quickly and should be immaculately clean.
The low boiling point is an excellent feature for heat transfer,
too, since it boils from the heat of the processor (ebullient
cooling). This change of state absorbs a lot of energy, making it
very effective at transferring heat away from the processor. The
vapor can then rise and condense on a heat exchanger with a chilled
water heat exchanger, where it again transfers a lot of heat through
a change of state.
Prentice
On 11/05/2018 06:30 PM, Stu Midgley wrote:
I refute both these claims.
You DO want to run your boards immersed in coolant. It works
wonderfully well, is easy to live with, servicing is easy... and
saves you almost 1/2 your power bill.
People are scared of immersion cooling, but it isn't that difficult
to live with. Some things are harder but other things are way
easier. In total, it balances out.
Also, given the greater reliability of components you get, you do
less servicing.
If you haven't lived with it, you really have no idea what you are
missing.
Serviceability is NOT challenging.
You really do NOT want to run boards immersed in coolant - yeah,
there's folks doing it at HPC scale
Whatever the coolant, it leaks, it oozes, it gets places you don't
want it to go. And serviceability is challenging. You need to pull
the "wet" boards out, or you need to connect and disconnect fluid
connectors, etc. If you're in an environment where you can manage
that (or are forced into it by necessity), then you can do it.
--
Dr Stuart Midgley
sdm...@gmail.com
_______________________________________________
Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin
Computing
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
