Hi Mark,
For my next cluster room, I am hoping to use 'liquid cooled' racks make by
Knurr (CoolTherm, http://www.thermalmanagement.de/). The scale is 67 racks
x 7.5kW heat removal per rack (36 U usable per rack).
7.5 KW/rack isn't much; are you designing low-power nodes?
Yup -- I guess so. For example our current cluster nodes are dual core
Opteron 175s (Supermicro H8SSL-i motherboards). They cost about $1200 for
2 x 2.2 GHz, with 1GB per core, and use 180 W under load (per 1U).
I will have enough rack space (67 x 34 U, per 500kW power and cooling)
that if I want to use nodes that dissipate 300 W per node then I will
simply limit myself to 25 nodes per rack.
The racks have front and back doors that close, and contains fans in the
back which circulate air through a heat exchanger located in the bottom of
the rack. The heat exchanger transfers the heat into chilled water.
APC has something similar with the cooling on the side.
Can you give a positive or negative opinion about either the Knurr or APC
racks? Have you used them yourself in a system?
The advantages of this are that it is quiet, and that you don't need
vertical height for underfloor ducting or overhead hot air removal.
Disadvantages are cost, potential difficulty of working within the rack.
and loss of one rack of cooling capacity if you open both the front and
back doors.
I'm pretty skeptical of the sealed-pod approach, since it seems to
multiply the number of parts, create access issues, doesn't seem to
actually save on space, etc.
It does save on vertical space, and it reduces noise to the level where
you can work comfortably in the room.
I've also been burned by cold water cooling, so to speak (assuming you
don't have your own, well-controlled CW plant.)
I'll have my own self-contained plant, designed by German engineers. They
seem to know their stuff.
I would definitely consider a normal big-chillers approach _with_
back-of-rack CW boosters (heat exchangers).
Can you provide a URL or recommendation for these back-of-rack CW
boosters?
and I'd definitely consider creative layouts of racks and chillers (for
instance, I'm not crazy about the hot/cold-aisle approach - something
W-shaped would be better for my ~50-rack machineroom. or even just
lining all the hot racks up against the chillers along one wall.)
(In this case I have access to 'building' funds that can not be used to buy
more cpus, so the cost issue is not important.)
are you sure you can't just be clever about laying out normal front-to-back
racks? possibly with back-of-rack heat-exchangers?
I'd like a URL for the back-of-rack heat exchangers, please!
(PS: though that approach still sounds noisy!)
Cheers,
Bruce
_______________________________________________
Beowulf mailing list, Beowulf@beowulf.org
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
