Re: [Beowulf] Servers Too Hot? Intel Recommends a Luxurious Oil Bath

2012-09-04 Thread Lux, Jim (337C)
Perhaps this is why Cray was in Minnesota? On 9/4/12 9:38 PM, "Joe Landman" wrote: >On 09/05/2012 12:28 AM, Lux, Jim (337C) wrote: > >> But as long as we're talking quarrys and such, what about the scheme of >> building a big pit to fill full of ice during the winter, and melting it >> during th

Re: [Beowulf] Servers Too Hot? Intel Recommends a Luxurious Oil Bath

2012-09-04 Thread Lux, Jim (337C)
Transformer oil isn't all that big a deal.. You buy it in 5 gallon pails, 55 gallon drums, tank truck lots. Runs about $3-4/gallon in large quantities (which is what ALL liquids cost, it seems.. Transportation costs are the dominant component. Sulfuric acid is pretty much the same price as min

Re: [Beowulf] Servers Too Hot? Intel Recommends a Luxurious Oil Bath

2012-09-04 Thread Joe Landman
On 09/05/2012 12:28 AM, Lux, Jim (337C) wrote: > But as long as we're talking quarrys and such, what about the scheme of > building a big pit to fill full of ice during the winter, and melting it > during the summer. (assuming you are in a > less-than-wonderful-un-California-like climate where thi

Re: [Beowulf] Servers Too Hot? Intel Recommends a Luxurious Oil Bath

2012-09-04 Thread Lux, Jim (337C)
On 9/4/12 3:46 PM, "Vincent Diepeveen" wrote: > >Uranium is dirt cheap as well, and i don't refer to mox fuel then, >yet to the actual >uranium you (not me of course) can dig up so easily in say Australia >or Syria; >dirt cheap if we look at what a central needs a year versus central >constructi

Re: [Beowulf] Servers Too Hot? Intel Recommends a Luxurious Oil Bath

2012-09-04 Thread Lux, Jim (337C)
On 9/4/12 11:54 AM, "Ellis H. Wilson III" wrote: >On 09/04/2012 12:54 PM, Lux, Jim (337C) wrote: >> Hydrogen is cheaper than He and works even better. Just make sure you >>don't have any air leaks in (i.e. keep a bit of positive pressure). For >>the "server farm in a container" model, this wo

Re: [Beowulf] Servers Too Hot? Intel Recommends a Luxurious Oil Bath

2012-09-04 Thread Lux, Jim (337C)
A notable, but non-obvious, hazard of large oil cooled systems (or oil tanks, for that matter) is that you don't float in them. Fall into a tank of water, and you're positively buoyant, so all you have to do is keep your head above water. Fall into a tank of oil with SG=0.8 and you sink like the p

Re: [Beowulf] NFS over RDMA small block DIRECT_IO corruption

2012-09-04 Thread Christopher Samuel
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/09/12 13:13, Douglas J. Trainor wrote: > p.s. could a list administrator please somehow delete > trai...@transborder.net from the list? Hmm, it would appear not, whilst it appears in the list of subscribers when I click the button next to it t

Re: [Beowulf] NFS over RDMA small block DIRECT_IO corruption

2012-09-04 Thread Christopher Samuel
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/09/12 13:13, Douglas J. Trainor wrote: > the problem is the ssl cert issued by Trond Myklebust expired July > 23… Indeed, but that doesn't mean it's serving malware. cheers! Chris - -- Christopher SamuelSenior Systems Administrator

Re: [Beowulf] NFS over RDMA small block DIRECT_IO corruption

2012-09-04 Thread Douglas J. Trainor
the problem is the ssl cert issued by Trond Myklebust expired July 23… douglas p.s. could a list administrator please somehow delete trai...@transborder.net from the list? the mailman database is corrupted/denormalized [or something], and this is the zombie address that can only receiv

Re: [Beowulf] NFS over RDMA small block DIRECT_IO corruption

2012-09-04 Thread Christopher Samuel
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 04/09/12 21:38, Vincent Diepeveen wrote: > That link gets reported here as spreading spyware? Looks fine to me.. http://www.google.com/safebrowsing/diagnostic?site=bugzilla.linux-nfs.org - -- Christopher SamuelSenior Systems Administra

Re: [Beowulf] Servers Too Hot? Intel Recommends a Luxurious Oil Bath

2012-09-04 Thread Peter St. John
Wiki tells me that the flash point of Transformer Oil (a type of mineral oil) is 140 C; does that sound safe in a server room? I'm a worse chemist than I am a physicist so I can't tell if you're serious about OSHA not liking mineral oil in server rooms (I'm **pretty** sure you're not serious about

Re: [Beowulf] Servers Too Hot? Intel Recommends a Luxurious Oil Bath

2012-09-04 Thread Vincent Diepeveen
On Sep 4, 2012, at 11:04 PM, Ellis H. Wilson III wrote: > On 09/04/2012 04:59 PM, Joe Landman wrote: >> On 09/04/2012 12:54 PM, Lux, Jim (337C) wrote: >>> Hydrogen is cheaper than He and works even better. Just make >>> sure you don't have any air leaks in (i.e. keep a bit of positive >>> pr

Re: [Beowulf] Servers Too Hot? Intel Recommends a Luxurious Oil Bath

2012-09-04 Thread Vincent Diepeveen
On Sep 4, 2012, at 10:50 PM, Robert G. Brown wrote: > On Tue, 4 Sep 2012, Lux, Jim (337C) wrote: > >> Hydrogen is cheaper than He and works even better. Just make sure >> you don't have any air leaks in (i.e. keep a bit of positive >> pressure). For the "server farm in a container" model, t

Re: [Beowulf] Servers Too Hot? Intel Recommends a Luxurious Oil Bath

2012-09-04 Thread Ellis H. Wilson III
On 09/04/2012 04:59 PM, Joe Landman wrote: > On 09/04/2012 12:54 PM, Lux, Jim (337C) wrote: >> Hydrogen is cheaper than He and works even better. Just make sure you don't >> have any air leaks in (i.e. keep a bit of positive pressure). For the >> "server farm in a container" model, this would w

Re: [Beowulf] Servers Too Hot? Intel Recommends a Luxurious Oil Bath

2012-09-04 Thread Joe Landman
On 09/04/2012 12:54 PM, Lux, Jim (337C) wrote: > Hydrogen is cheaper than He and works even better. Just make sure you don't > have any air leaks in (i.e. keep a bit of positive pressure). For the > "server farm in a container" model, this would work just fine.. leaks would > just float up int

Re: [Beowulf] Servers Too Hot? Intel Recommends a Luxurious Oil Bath

2012-09-04 Thread Robert G. Brown
On Tue, 4 Sep 2012, Lux, Jim (337C) wrote: > Hydrogen is cheaper than He and works even better. Just make sure you don't > have any air leaks in (i.e. keep a bit of positive pressure). For the > "server farm in a container" model, this would work just fine.. leaks would > just float up into t

Re: [Beowulf] Servers Too Hot? Intel Recommends a Luxurious Oil Bath

2012-09-04 Thread Nathan Moore
> > I know we've been taking things to the uber-scale level with this > conversation, but does anyone have suggestions for small (homebrew > Beowulf) clusters? If you live in a cold climate, there's not much difference between a rack of servers and baseboard electric heat - just limit your dataru

Re: [Beowulf] Servers Too Hot? Intel Recommends a Luxurious Oil Bath

2012-09-04 Thread Vincent Diepeveen
On Sep 4, 2012, at 8:54 PM, Ellis H. Wilson III wrote: > On 09/04/2012 12:54 PM, Lux, Jim (337C) wrote: >> Hydrogen is cheaper than He and works even better. Just make sure >> you don't have any air leaks in (i.e. keep a bit of positive >> pressure). For the "server farm in a container" mod

Re: [Beowulf] Servers Too Hot? Intel Recommends a Luxurious Oil Bath

2012-09-04 Thread Joe Landman
On 09/04/2012 02:56 PM, Douglas Eadline wrote: > > Of course those massive Zetta scale systems will live in huge > multi-story oil tanks that have been placed in old quarrys, > which provide bedrock support for the tank and a geothermal > heat sink. The sysadmins must operate oil swimming robots an

Re: [Beowulf] Servers Too Hot? Intel Recommends a Luxurious Oil Bath

2012-09-04 Thread Douglas Eadline
Of course those massive Zetta scale systems will live in huge multi-story oil tanks that have been placed in old quarrys, which provide bedrock support for the tank and a geothermal heat sink. The sysadmins must operate oil swimming robots and even don oil scuba suits to service hardware. Don't fo

Re: [Beowulf] Servers Too Hot? Intel Recommends a Luxurious Oil Bath

2012-09-04 Thread Ellis H. Wilson III
On 09/04/2012 12:54 PM, Lux, Jim (337C) wrote: > Hydrogen is cheaper than He and works even better. Just make sure you don't > have any air leaks in (i.e. keep a bit of positive pressure). For the > "server farm in a container" model, this would work just fine.. leaks would > just float up int

Re: [Beowulf] Servers Too Hot? Intel Recommends a Luxurious Oil Bath

2012-09-04 Thread Robert G. Brown
On Tue, 4 Sep 2012, John Hearns wrote: > As this article points out, helium is a finite resource. If all teams > were using it, there would > be no advantage to any one team and it would be wasteful of a resource. > If I'm nto wrong the US Government holds the biggest stock of helium, > somewhere

Re: [Beowulf] Servers Too Hot? Intel Recommends a Luxurious Oil Bath

2012-09-04 Thread Prentice Bisbal
On 09/03/2012 08:04 PM, Christopher Samuel wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On 03/09/12 23:15, Douglas Eadline wrote: > >> There was a previous discussion here. Curious to see Intel's >> interest in this tech. I assumed it is mostly used for edge cases. > It's going to c

Re: [Beowulf] Servers Too Hot? Intel Recommends a Luxurious Oil Bath

2012-09-04 Thread Lux, Jim (337C)
Hydrogen is cheaper than He and works even better. Just make sure you don't have any air leaks in (i.e. keep a bit of positive pressure). For the "server farm in a container" model, this would work just fine.. leaks would just float up into the atmosphere. @ 300K Air 26.2 mW/m*K He 156.7 H2 1

Re: [Beowulf] Servers Too Hot? Intel Recommends a Luxurious Oil Bath

2012-09-04 Thread Hearns, John
LN2 OTOH is very cheap, easy to make, and there is a potentially limitless supply (think very large atomospheric reservoir of gaseous N2 we could tap). Just need to replace heat sinks with conduction plates, and have LN2 run through heat exchangers in bottoms of racks. It won't short out elec

Re: [Beowulf] Servers Too Hot? Intel Recommends a Luxurious Oil Bath

2012-09-04 Thread atchley tds.net
On Tue, Sep 4, 2012 at 9:29 AM, John Hearns wrote: > On 4 September 2012 14:16, Robert G. Brown wrote: >> >> On a more serious note, one wonders why nobody has tried helium instead. >> No, silly, not liquid helium, helium gas. The reason they fill windows >> with argon is that it has around 2/3

Re: [Beowulf] Servers Too Hot? Intel Recommends a Luxurious Oil Bath

2012-09-04 Thread Joe Landman
On 09/04/2012 09:16 AM, Robert G. Brown wrote: > Helium, OTOH, has six times the thermal conductivity of air, and is > relatively inexpensive. The biggest downside I can think of is that it > requires a pretty good seal and thick walls to keep the slippery little > atoms from sliding right throug

Re: [Beowulf] Servers Too Hot? Intel Recommends a Luxurious Oil Bath

2012-09-04 Thread Vincent Diepeveen
I read a report somewhere some big CERN type projectmembers were warnng that other areas of science shouldn't waste helium, especially not for balloons, as when it escapes it leaves planet earth and they need it badly to cool down things and foresee a problem for the far future as what left pla

Re: [Beowulf] Servers Too Hot? Intel Recommends a Luxurious Oil Bath

2012-09-04 Thread John Hearns
On 4 September 2012 14:16, Robert G. Brown wrote: > > On a more serious note, one wonders why nobody has tried helium instead. > No, silly, not liquid helium, helium gas. The reason they fill windows > with argon is that it has around 2/3 the thermal conductivity of air, > and hence is a better i

Re: [Beowulf] Servers Too Hot? Intel Recommends a Luxurious Oil Bath

2012-09-04 Thread Robert G. Brown
On Mon, 3 Sep 2012, Lux, Jim (337C) wrote: > I'll bet they have to change it more often than that. This isnt something > like a pole transformer. Absolutely. Think of what you can do with a big vat of hot oil handy in the workspace. Buffalo Wings. French Fries. Chicken. Fish. The reason nob

Re: [Beowulf] NFS over RDMA small block DIRECT_IO corruption

2012-09-04 Thread Joe Landman
On 09/04/2012 06:57 AM, hol...@th.physik.uni-frankfurt.de wrote: > FYI > > https://bugzilla.linux-nfs.org/show_bug.cgi?id=228 > > Careful with your block sizes when doing DIRECT_IO people :) Direct IO should work only with 4k and multiple of 4k block sizes. It does page-at-a-time IO, and doesn't

Re: [Beowulf] NFS over RDMA small block DIRECT_IO corruption

2012-09-04 Thread Vincent Diepeveen
That link gets reported here as spreading spyware? On Sep 4, 2012, at 12:57 PM, hol...@th.physik.uni-frankfurt.de wrote: > FYI > > https://bugzilla.linux-nfs.org/show_bug.cgi?id=228 > > Careful with your block sizes when doing DIRECT_IO people :) > > Ta, > > Andrew > > ___

[Beowulf] NFS over RDMA small block DIRECT_IO corruption

2012-09-04 Thread holway
FYI https://bugzilla.linux-nfs.org/show_bug.cgi?id=228 Careful with your block sizes when doing DIRECT_IO people :) Ta, Andrew ___ Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or un