It’s really a matter of what technology you want. KNL we run at about 320W so 800/320*3=7.5TF V100 we run at about 300W so 800/300*6=16TF
BUT our KNL’s sit turbo boosted nearly all the time so more like 3.5TF each. On Wed, 11 Mar 2020 at 7:04 pm, Scott Atchley <e.scott.atch...@gmail.com> wrote: > Hi Stu, > > The rolling weight is only an issue when moving equipment during > installation/removal. It causes point loads and we typically lay steel > plate down to spread the load over multiple tiles. > > What is your performance density (FLOPS/ft^2) in Houston if you do not > mind me asking? > > Scott > > On Tue, Mar 10, 2020 at 8:30 PM Stu Midgley <sdm...@gmail.com> wrote: > >> Immersion cooling makes a lot of sense :) >> >> We run it on the 21st floor of a building in Kuala Lumpur, on the 1st >> floor in Perth and on a slab-on-ground in Houston. >> >> The tanks+fluid are light. When full of equipment, about 1.2 tonnes >> spread over about 2m2. >> >> In Houston, we have the lowest rated raised floor - since the tanks >> spread the load across multiple tiles/floor stands, and there is no rolling >> weight (its spread evenly over 2m2) >> >> So ~ 2200lbs/10sqft ie. about 220lbs/sqft . We run a power density of >> 8.5kW/sqm (~800W/sqft) across our whole DC (which includes all the internal >> white space/CRAC space etc). >> >> We cool the whole facility with evaporation (compressor cooling is only >> for comfort cooling). >> >> We have hit a PUE of 1.045 in Houston... and 1.035 in Perth :) >> >> Come and have a look at our Houston DC :) >> >> >> >> On Wed, Mar 11, 2020 at 3:37 AM Scott Atchley <e.scott.atch...@gmail.com> >> wrote: >> >>> Hi everyone, >>> >>> I am wondering whether immersion cooling makes sense. We are most >>> limited by datacenter floor space. We can manage to bring in more power (up >>> to 40 MW for Frontier) and install more cooling towers (ditto), but we >>> cannot simply add datacenter space. We have asked to build new building and >>> the answer has been consistently "No." >>> >>> Summit is mostly water cooled. Each node has cold plates on the CPUs and >>> GPUs. Fans are needed to cool the memory and power supplies and is captured >>> by rear-door heart exchangers. It occupies roughly 5,600 ft^2. With 200 PF >>> of performance and 14 MW of power, that is 36 TF/ft^2 and 2.5 kW/ft^2. >>> >>> I am wondering what the comparable performance and power is per square >>> foot for the densest, deployed (not theoretical) immersion cooled systems. >>> Any ideas? >>> >>> To make the exercise even more fun, what is the weight per square foot >>> for immersion systems? Our data centers have a limit of 250 or 500 >>> pounds per square foot. I expect immersion systems to need higher loadings >>> than that. >>> >>> Thanks, >>> >>> Scott >>> _______________________________________________ >>> Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing >>> To change your subscription (digest mode or unsubscribe) visit >>> https://beowulf.org/cgi-bin/mailman/listinfo/beowulf >>> >> >> >> -- >> Dr Stuart Midgley >> sdm...@gmail.com >> > -- Dr Stuart Midgley sdm...@gmail.com
_______________________________________________ Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit https://beowulf.org/cgi-bin/mailman/listinfo/beowulf
