On 10/04/11 16:52, Greg Lindahl wrote: > On Tue, Oct 04, 2011 at 03:29:28PM -0400, Chris Dagdigian wrote: >> I'm largely with RGB on this one with the minor caveat that I think he >> might be undervaluing the insane economies of scale that IaaS providers >> like Amazon & Google can provide. > > cheap land and power hence cheap datacenter rents. And with only 750 > servers, we are already big enough to reap enough outright economy of > scale to make leasing our own servers in a rented datacenter cheaper > than renting everything from Amazon. > > The unique thing Amazon is providing is the ability to grow and shrink > your cluster. Your example of a company which wanted to run a bunch of > molecular dynamics computations in a short period of time is an > illustration of that.
On this note, does anyone know if there are prior works (either academic or publicly disclosed documentations of a company pursuing such a route) of people splitting their workload up into the "static" and "dynamic" portions and running them respectively on in-house and rented hardware? While I see this discussion time and time again go either one way or the other (google or amazon, if you will), I suspect for many companies if it were possible to "invisibly" extend their infrastructure into the cloud on an as-needed basis, it might be a pretty attractive solution. Put another way, there doesn't seem to be much sense in buying a couple more racks for just a short-term project that will result in those racks going silent afterwards. On the flipside, you probably have some fraction of the compute and data resources you need as it is, you just want it to run a little faster or need a little more scratch space/bandwidth. So renting an entire set of resources wouldn't be optimal either, since that will result in underutilization of the infrastructure at home. So just buy whatever fraction your missing from Amazon from a month and use some hacks to make it look like that hardware is right there next to your other stuff. Obviously this requires an embarrassingly parallel workload due to the locality dichotomy (or completely disjoint workloads). Another idea I had was just like solar energy, what if there was a way for you to build up credits for Amazon in the "day" and use them at "night"? I.E. put some Amazon software on your infrastructure that allows you them to use your servers as part of their "cloud" when you're not using your equipment at max, and when you do go peak it will automatically provision more and more Amazon leased resources on an as-needed basis and burn up those earned credits instead of "real money." Just some ideas I figured I'd put through the beo-blender to see if they hold any weight before actually pursuing them as research objectives. ellis _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
