Jonathan Aquilina wrote: > > It’s not that I need to cluster these vps’s I was just wondering if it > was possible. What puts me off about amazon is pricing. It seems a bit > pricy so to speak. >
MIT StarCluster (the open source stack that builds Grid Engine clusters on Amazon mentioned elsewhere in this thread) is able to leverage the AWS Spot Market and the potential savings off of the hourly EC2 rate is pretty enormous. Via Spot you can run servers for pennies an hour that traditionally sell for dollars-per-hour on the "normal" EC2 on-demand service. It would be very hard to beat that price on an internal infrastructure if one was honest about the fully loaded facility, energy and staffing costs. I ran 30-40 of Amazon's biggest and newest compute cluster instance types + 16 of their GPU nodes on the Spot Market and a few persistent license and control servers to manage the workload and I think our hourly cost was less than $30/hour. We only paid that rate for the duration of the simulation then tore everything down. Totally worth it and we ran at a scale that would not have been possible internally. I was originally leery of the Spot Market thinking that it would be too disruptive to have nodes potentially yanked from under me but it turns out the pricing history is easily accessible and you can see that over time the prices are usually super stable. I've been able to pay pennies for decent sized servers and run them for weeks at a time. However there are occasionally spikes and it's interesting to see how the arbitrage/bidding strategies work. I've been doing some low-importance analysis on a c1.medium server recently and I ran for a few days at just a few pennies an hour before my server was blown away due to the spot price rising higher than my max bid ($0.11) . Looking at the pricing history there was a brief 1-hour window of time when the spot price for a c1.medium instance type in region us-east-1b spiked to *several dollars* -- far higher than even the hourly on-demand rate that Amazon charges for the same server. One piece of advice is to avoid us-east-1b -- that is my personal default and I think many others do the same. Spot price variability seems less volatile in the other zones. My $.02! -Chris _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
