I suppose, we are talking about HPC here with already large infrastructure, so 
the “incremental pain” from having to deal with immersion cooling is probably 
smaller than the “incremental pain” from having to air cool a substantially 
larger facility – floor space costs money in many ways.  Having a rig to hoist 
electronics out of the vat (or drain the tank, or however) isn’t all that much 
different than having any other specialized support equipment –

If you had a few dozen processors, probably not worth it – get up to 1000 
nodes, and it winds up being small in comparison.

However, I think it is still pretty exotic – you guys have it figured out, and 
that’s probably part of your secret sauce – I wouldn’t recommend it for the 
“casual” or “small” HPC installation.

OTOH, if someone wants to try it, have at it – it does work really well from a 
thermal standpoint – moving liquid around is MUCH more efficient than moving 
air around.



Jim Lux
(818)354-2075 (office)
(818)395-2714 (cell)

From: Stu Midgley [mailto:sdm...@gmail.com]
Sent: Monday, November 05, 2018 3:31 PM
To: Lux, Jim (337K) <james.p....@jpl.nasa.gov>
Cc: Jonathan Aquilina <jaquil...@eagleeyet.net>; ghenr...@gmail.com; Beowulf 
List <beowulf@beowulf.org>
Subject: Re: [Beowulf] More about those underwater data centers

I refute both these claims.

You DO want to run your boards immersed in coolant.  It works wonderfully well, 
is easy to live with, servicing is easy... and saves you almost 1/2 your power 
bill.

People are scared of immersion cooling, but it isn't that difficult to live 
with.  Some things are harder but other things are way easier.  In total, it 
balances out.

Also, given the greater reliability of components you get, you do less 
servicing.

If you haven't lived with it, you really have no idea what you are missing.


Serviceability is NOT challenging.



You really do NOT want to run boards immersed in coolant - yeah, there's folks 
doing it at HPC scale

Whatever the coolant, it leaks, it oozes, it gets places you don't want it to 
go. And serviceability is challenging. You need to pull the "wet" boards out, 
or you need to connect and disconnect fluid connectors, etc.  If you're in an 
environment where you can manage that (or are forced into it by necessity), 
then you can do it.
--
Dr Stuart Midgley
sdm...@gmail.com<mailto:sdm...@gmail.com>
_______________________________________________
Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit 
http://www.beowulf.org/mailman/listinfo/beowulf

Reply via email to