The recent digests that I am getting are quite interesting (bad google) and I have a question.
What I'd like to know is, is it possible to have a our history captured in its entirety so that none of the future generations have to run around (like Hari Seldon) because information from waaaay back is corrupt and not take care of? Do you guys know of any existing sources that you can point me to? Is this under distributed systems or under compression algorithms? Any other two cents on this is welcome! Sent from my BlackBerry -----Original Message----- From: [email protected] Sender: [email protected] Date: Wed, 27 Oct 2010 09:36:13 To: <[email protected]> Reply-To: [email protected] Subject: Beowulf Digest, Vol 80, Issue 22 Send Beowulf mailing list submissions to [email protected] To subscribe or unsubscribe via the World Wide Web, visit http://www.beowulf.org/mailman/listinfo/beowulf or, via email, send a message with subject or body 'help' to [email protected] You can reach the person managing the list at [email protected] When replying, please edit your Subject line so it is more specific than "Re: Contents of Beowulf digest..." Today's Topics: 1. RE: how Google warps your brain (Bill Rankin) 2. RE: how Google warps your brain (Douglas Eadline) 3. RE: Anybody using Redhat HPC Solution in their Beowulf (Hearns, John) 4. Re: Anybody using Redhat HPC Solution in their Beowulf (Ellis H. Wilson III) 5. Re: Anybody using Redhat HPC Solution in their Beowulf (Kilian CAVALOTTI) 6. RE: Anybody using Redhat HPC Solution in their Beowulf (Lux, Jim (337C)) ---------------------------------------------------------------------- Message: 1 Date: Tue, 26 Oct 2010 14:54:43 +0000 From: Bill Rankin <[email protected]> Subject: RE: [Beowulf] how Google warps your brain To: Beowulf Mailing List <[email protected]> Cc: "Robert G. Brown" <[email protected]> Message-ID: <[email protected]> Content-Type: text/plain; charset="us-ascii" Heading completely off-topic now, but the area of digital media and long-term archival/retrieval is something that I find very interesting. I'll leave it to Rob to somehow eventually tie this back into a discussion of COTs technology and HPC. > > It's interesting: I just got an iPad a few weeks ago, mostly as a > > reader/web-browser device, and I've been reading a variety of > > out-of-copyright works: H. Rider Haggard, Joseph Conrad, Mark Twain. > Thank > > you Gutenberg Project! > > It is awesome, isn't it? Amazon also carries many of the out-of-copyright works in their Kindle store for $0 (and gives credit to Gutenburg to a small extent). It was nice to be able to go pickup things like the Sherlock Holmes series, Homer's Illiad and some of Einstein's works (which I don't pretend to understand) and have them downloaded via 3G on Amazon's dime. I will say that because of this I tend to overlook their rather high (IMHO) price on current digital content and have probably purchased more e-books overall as a result. > > And, since I am sitting/lying here with a very sore back from moving boxes > > of books around this weekend looking for that book that I *know* is in there > > somewhere, the prospect of some magic box that would scan all my books into > > a format usable into eternity would be quite nice. I might even think that > > a personal "print on demand" would be nice that could generate a cheap/quick > > copy for reading in bed(yes, the iPad and Kindle, etc., are nice, but > > there's affordances provided by the paper edition that is nice.. But I don't > > need hardcover or, even, any cover..) There is just *something* about paper, isn't there? And while I don't have a library to the extent of RGBs or others, I do like having some books around (glancing at the two bookshelves in my office). On the other hand, I still have boxes of books sitting around unopened since we moved house 4-5 years ago. I certainly need a purge, lest I end up on one of those "hoarding" shows that seem to be popular as of late. At some point, I have to ask myself if I really *need* to have a old beat-up, falling apart copy of "Voyage of the Space Beagle" laying around. > > (or, even better, a service that has scanned all the books for me, e.g. > > Google, and that upon receiving some proof of ownership of the physical > > book, lets me have an electronic copy of the same... I'd gladly pay some > > nominal fee for such a thing, providing it wasn't for some horrible locked, > > time limited format which depends on the original vendor being in business > > 20 years from now. I also recognize the concern about how "once in digital > > form, copying becomes very cheap" which I think is valid. A scanning service would be wonderful for a lot of the books I have, mainly those I view as reference-type material. For current reference material, Safari Books Online has a reasonable usage model that allows for making hardcopy of their online content. Now if there was only a simple way to transcribe the same content for download to my Kindle I would be set (something beyond the OCR+PDF approach, which is awkward and inconsistent). > What a killer idea. Acceptable use, doggone it! I'd ship them books > by the boxful in exchange for a movable (even DRM controlled) image, a la > Ipod music. I just don't want to rebuy them, like I've now bought most > of my music collection TWICE (vinyl and CD). [let's not get started about vinyl collections - that's a whole 'nother set of unopened boxes] The problem is that many of the media houses are still waging an underground war on Fair Use, despite the legal decisions handed down by the courts. As an example, I recently had a email exchange with one of the customer service people at a major network. I was trying to locate additional interview footage from when my brother-in-law was on a certain hour-long Sunday evening news show. This person informed me that I did not have their "permission" to recorded the over-the-air broadcast of the show and burn it on a DVD to give to my sister, so what I was doing was not legal. This was news to me, since this usage model was clearly defined as permissible by the Supreme Court many years ago in the Sony v. Universal "Betamax Case". While the market for online music, video and written works have forced the various publishers to acknowledge to the need to provide content in digital form, to a great extent they had to be dragged kicking and screaming into the 21st century. A lot of progress has been made but there is still a lot of resistance towards efforts to open up availability and access even further. I would like see a service where I could take bins of old books to a used book store and somehow get credits towards the purchase of e-books online. I think that could break me of my paperback hoarding habit pretty quickly. -bill ------------------------------ Message: 2 Date: Tue, 26 Oct 2010 10:59:25 -0400 (EDT) From: "Douglas Eadline" <[email protected]> Subject: RE: [Beowulf] how Google warps your brain To: "Hearns, John" <[email protected]> Cc: [email protected], "Robert G. Brown" <[email protected]> Message-ID: <[email protected]> Content-Type: text/plain;charset=iso-8859-1 <Seinfeld> Not that there is anything wrong with that. </Seinfeld > > > As usual, a highly insightful post from RGB. > > > >> a) Multiple copies. Passenger pigeons may be robust, but once the > number of copies drops below a critical point, they are gone. E. Coli > we will always have >> with us (possibly in a constantly changing form) because there are so > very many copies, so very widely spread. > > I probably shouldn't mention Wikileaks here... > >> >> At the moment, the internet has if anything VASTLY INCREASED a, b and > c >> for every single document in the public domain that has been ported > to, >> e.g. Project Gutenberg. >> >> Right now, I'm sitting on a cache of "Saint" books, by Leslie > Charteris >> (who was a great favorite of mine growing up and still is). >> >> Nobody is going to reprint the Saint stories. They are a gay fantasy >> from another time, > > Simon Templar? Gay? Cough. > > Next you will be telling me that there are gay undertones in Top Gun, > the film with the sexiest astrophysicist ever. > > >> might well last to the end of civilization. Replicate them a few >> million times, PERPETUATE them from generation to generation by >> renewing >> the copies, and backing them up, and recopying them in formats where >> they are still useful. > > The cloud backup providers will be keeping copies of data on > geographically spread sites. > However, we should at this stage be asking what are the mechanisms for > cloud storage companies > for > *) living wills - what happens when the company goes bust > > *) what are the strategies for migrating the data onto new storage > formats > > >> >> Or, to put it differently, suppose every single human on the planet > had >> access to the modern equivalent of Diophantus's Arithmetica on their >> computer, their Kindle, their Ipad > I believe that was the original intent for the Web. Still under > development! > > > The contents of this email are confidential and for the exclusive use of > the intended recipient. If you receive this email in error you should not > copy it, retransmit it, use it or disclose its contents but should return > it to the sender immediately and delete your copy. > > _______________________________________________ > Beowulf mailing list, [email protected] sponsored by Penguin Computing > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf > > -- > This message has been scanned for viruses and > dangerous content by MailScanner, and is > believed to be clean. > -- Doug -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. ------------------------------ Message: 3 Date: Tue, 26 Oct 2010 09:16:47 +0100 From: "Hearns, John" <[email protected]> Subject: RE: [Beowulf] Anybody using Redhat HPC Solution in their Beowulf To: "Ellis H. Wilson III" <[email protected]>, <[email protected]> Message-ID: <68a57ccfd4005646957bd2d18e60667b12154...@milexchmb1.mil.tagmclarengroup.com> Content-Type: text/plain; charset="us-ascii" > I don't think you could find a statement more orthogonal to the spirit > of the Beowulf list than, "Please, please don't "roll your own" > system..." Isn't Beowulfery about the drawing together of inexpensive > components in an intelligent fashion suited just for your particular > application while using standardized (and thereby cheap by the law of > scale) hardware? I'm not suggesting Richard build his own NIC - but > there is nothing wrong with using even a distribution of Linux not > intended for HPC (so long as you're smart about it) and picking and > choosing the software (queuing managers, tracers, etc) he finds works > best. > > Also, I would argue if a company is selling you an HPC solution, it's > either: > 1. A true Beowulf in terms of using COTS hardware, in which case you > are > likely getting less than your money is worth or Ellis, I am going to politely disagree with you - now there's a surprise! I have worked as an engineer for two HPC companies - Clustervision and Streamline. My slogan phrase on this issue is "Any fool can go down PC World and buy a bunch of PCs" By that I mean that CPU is cheap these days, but all you will get is a bunch of boxes on your loading bay. As you say, and you are right, you then have the option of installing Linux plus a cluster management stack and getting a cluster up and running. However, as regards price, I would say that actually you will be paying very, very little premium for getting a supported, tested and pre-assembled cluster from a vendor. Academic margins are razor thin - the companies are not growing fat over academic deals. They also can get special pricing from Intel/AMD if the project can be justified - probably ending up at a price per box near to what you pay at PC World. Or take (say) rack top switches. Do you want to have a situation where the company which supports your cluster has switches sitting on a shelf, so when a switch fails someone (me!) is sent out the next morning to deliver a new switch in a box, cable it in and get you running? Or do you want to deal direct with the returns department at $switch vendor, or even (shudder) take the route of using the same switches as the campus network - so you don't get to choose on the basis of performance or suitability, but just depend on the warm and fuzzies your campus IT people have. We then come to support - say you buy that heap of boxes from a Tier 1 - say it is the same company your campus IT folks have a campus wide deal with. You'll get the same type of support you get for general servers running Windows - and you'll deal with first line support staff on the phone every time. Me, I've been there, seen there, done it with tier 1 support like that. As a for instance, HPC workloads tend to stress the RAM in a system, and you get frequent ECC errors on a young system as it is bedding in. Try phoning support every time a light comes on, and get talked through the "have you run XXX diagnostic", it soon gets wearing. Before Tier 1 companies cry foul, of course both the above companies and all other cluster companies integrate Tier 1 servers - but that is a different scenario from getting boxes delivered through your campus agreement with $Tier1. The contents of this email are confidential and for the exclusive use of the intended recipient. If you receive this email in error you should not copy it, retransmit it, use it or disclose its contents but should return it to the sender immediately and delete your copy. ------------------------------ Message: 4 Date: Tue, 26 Oct 2010 12:09:12 -0400 From: "Ellis H. Wilson III" <[email protected]> Subject: Re: [Beowulf] Anybody using Redhat HPC Solution in their Beowulf To: "Hearns, John" <[email protected]> Cc: [email protected] Message-ID: <[email protected]> Content-Type: text/plain; charset=ISO-8859-1; format=flowed On 10/26/10 04:16, Hearns, John wrote: > I have worked as an engineer for two HPC companies - Clustervision and > Streamline. > My slogan phrase on this issue is "Any fool can go down PC World and buy > a bunch of PCs" Well if you are buying PCs in bulk at retail pricing, you are a fool anyway. Plus most PC World PCs won't have ECC RAM so I wasn't really referring to those as few of us tolerate random bit flips. > However, as regards price, I would say that actually you will be paying > very, very little premium > for getting a supported, tested and pre-assembled cluster from a vendor. > Academic margins are razor thin - the companies are not growing fat over > academic deals. > They also can get special pricing from Intel/AMD if the project can be > justified - probably ending > up at a price per box near to what you pay at PC World. Again, not comparing PC World to Tier 1 bulk purchases. I'm comparing Tier 1 bulk purchases w/o an OS (so you can DIY) with specialized HPC vendor purchases where you don't have to DIY. Even then, perhaps it breaks even the first year if you get a very, very good deal from the HPC vendor. However, to get the deal you are probably contracted into four or five years of support and when considering HPC, involving more humans are the fastest way to get a really inefficient and expensive cluster. After the first year and up until the lifetime of the cluster involving human support annually will add a large cost overhead you have to account for at the beginning (and probably buy less hardware because of which). > Or take (say) rack top switches. Do you want to have a situation where > the company which supports your cluster > has switches sitting on a shelf, so when a switch fails someone (me!) is > sent out the next morning to deliver > a new switch in a box, cable it in and get you running? That's probably a hell of a lot faster than waiting on a vendor to get you a new switch through some RMA process. Plus you know the cabling is done right :). Optimally IMHO, in university setups physical scientists create the need for HPC. These types shouldn't (as Kilian mentions) need to inherit all of the responsibilities and overheads of cluster management to use one (or pay cluster vendors annually for support). They should simply walk over to the CS department, find system guys (who would probably drool over the potential of administering a reasonably sized cluster) and work out an agreement where the physical science types can "just use it" and the systems/CS guys administer it and can once in a while trace workloads, test new load balancing mechanisms, try different kernel settings for performance, etc. This way the physical scientists get their work done on a well supported HPC system for no extra cash and computer scientists get great, non-toy traces and workloads to further their own research. Both parties win. Now in organizations that don't have a CS department I agree that HPC vendors are the way to go. ellis ------------------------------ Message: 5 Date: Tue, 26 Oct 2010 11:18:56 +0200 From: Kilian CAVALOTTI <[email protected]> Subject: Re: [Beowulf] Anybody using Redhat HPC Solution in their Beowulf To: "Ellis H. Wilson III" <[email protected]> Cc: [email protected] Message-ID: <[email protected]> Content-Type: text/plain; charset=UTF-8 Hi, On Tue, Oct 26, 2010 at 1:00 AM, Ellis H. Wilson III <[email protected]> wrote: > Also, I would argue if a company is selling you an HPC solution, it's > either: > 1. A true Beowulf in terms of using COTS hardware, in which case you are > likely getting less than your money is worth or Well, depends on how you value your time and the required expertise to put all those COTS and OSS pieces together to make them run smoothly and efficiently. Most scientists and HPC systems users are not professional sysadmins (which is good, they have a job to do), and the value of trained, experienced, skilled individuals who can put together a reliable and useful HPC system is sometimes overlooked (ie. undervalued). I agree with your later statement, though: > I personally don't think the "market for cluster vendors" is [...] > the Beowulf list. Cheers, -- Kilian ------------------------------ Message: 6 Date: Wed, 27 Oct 2010 09:32:43 -0700 From: "Lux, Jim (337C)" <[email protected]> Subject: RE: [Beowulf] Anybody using Redhat HPC Solution in their Beowulf To: "Ellis H. Wilson III" <[email protected]>, "Hearns, John" <[email protected]> Cc: "[email protected]" <[email protected]> Message-ID: <ece7a93bd093e1439c20020fbe87c47fedd29f9...@altphyembevsp20.res.ad.jpl> Content-Type: text/plain; charset="us-ascii" > > Optimally IMHO, in university setups physical scientists create the need > for HPC. These types shouldn't (as Kilian mentions) need to inherit all > of the responsibilities and overheads of cluster management to use one > (or pay cluster vendors annually for support). They should simply walk > over to the CS department, find system guys (who would probably drool > over the potential of administering a reasonably sized cluster) and work > out an agreement where the physical science types can "just use it" and > the systems/CS guys administer it and can once in a while trace > workloads, test new load balancing mechanisms, try different kernel > settings for performance, etc. This way the physical scientists get > their work done on a well supported HPC system for no extra cash and > computer scientists get great, non-toy traces and workloads to further > their own research. Both parties win. > I don't know about this model. This is like developing software on prototype hardware. The hardware guys and gals keep wanting to change the hardware, and the software developers complain that their software keeps breaking, or that the hardware is buggy (and it is). The computational physics and computational biology guys get to work on cool, nifty stuff to push their dissertation forward by using a hopefully stable computational platform. But I don't think the CS guys would drool over the possibility of administering a cluster. The CS guys get to be sysadmin/maintenance types...not very fun for them, and not the kind of work that would work for their dissertation. Now, if the two groups were doing research on new computational methods (what's the best way to simulate X) perhaps you'd get a collaboration. ------------------------------ _______________________________________________ Beowulf mailing list [email protected] http://www.beowulf.org/mailman/listinfo/beowulf End of Beowulf Digest, Vol 80, Issue 22 *************************************** _______________________________________________ Beowulf mailing list, [email protected] sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
