Mark Hahn wrote:
I don't buy the argument that the winning case is packaging up a VM with
all your software. If you really are unable to build the required
software stack for a given cluster and its OS, I think using something
you're right, but only for narrow-function clusters. suppose you h
then why not just run vm's on the host. also then in that case would it be
possible to point pxe and tell it when booting the nodes which image to use?
On Fri, Jan 29, 2010 at 10:55 PM, Ashley Pittman wrote:
>
> On 26 Jan 2010, at 19:37, Paul Van Allsburg wrote:
> > Ashley Pittman wrote:
> >> On
On 26 Jan 2010, at 19:37, Paul Van Allsburg wrote:
> Ashley Pittman wrote:
>> On 25 Jan 2010, at 15:28, Jonathan Aquilina wrote:
>>> has anyone tried clustering using xen based vm's. what is everyones take on
>>> that? its something that popped into my head while in my lectures today.
>>>
>>
On 28 Jan 2010, at 4:23 pm, Gavin Burris wrote:
Sorry, I'm not drinking the virtualization/cloud koolaid. I'd love to
have everything abstracted and easy to manage, but I find
standardizing
on an OS or two and keeping things as stock as possible is easier, and
cheaper to manage at this poin
Two staff couldn't handle 2k users and 100 of departments, or that much
hardware.
so you say. my example is a scaled down version of actual
numbers of my organization. of course, much depends on how
you define "user" (logged in right now or "getent passwd | wc -l"?)
or for that matter how y
Two staff couldn't handle 2k users and 100 of departments, or that much
hardware. Answering tickets or emails alone would be overwhelming.
Building/maintaining the VMs, or training/document/helping the
departments to build their own VMs is a monumental task in and of
itself. A more realistic numb
On 28 Jan 2010, at 3:10 pm, Mark Hahn wrote:
I don't buy the argument that the winning case is packaging up a VM
with
all your software. If you really are unable to build the required
software stack for a given cluster and its OS, I think using
something
you're right, but only for narrow
im just wondering now would for instance a head node be of any use running
virtualized guest os's or does the head node need to not share the hardware
with other os's
well, the HA-ish motive for VMs has some application to the admin
portions of even a pure HPC clustre. for instance, your jobs m
I don't buy the argument that the winning case is packaging up a VM with
all your software. If you really are unable to build the required
software stack for a given cluster and its OS, I think using something
you're right, but only for narrow-function clusters. suppose you have
a cluster use
On 1/27/2010 10:30 AM, Jonathan Aquilina wrote:
so basically what your saying is something along the lines of a
rendering cluster would be a good candidate for this?
I'm saying nothing about how a virtualized cluster could
or should be used. I'm only commenting about how a virtualized
cluster m
so basically what your saying is something along the lines of a rendering
cluster would be a good candidate for this?
On Wed, Jan 27, 2010 at 6:31 PM, Jon Forrest wrote:
> At a recent Rocks clustering user's group
> meeting the recent addition of Rocks support of
> Xen-based virtual clusters cam
At a recent Rocks clustering user's group
meeting the recent addition of Rocks support of
Xen-based virtual clusters came up. Some
of the same questions recently raised on this
list were discussed there.
One justification for virtual clusters that I
hadn't thought of was discussed. This only appl
thanks for all yoru responses. i admit i dont have the money at the moment
or a job to get my hands dirty with hpc. im planning in the future to setup
a rendering cluster. i appreciate all the feed back here.
im just wondering now would for instance a head node be of any use running
virtualized gu
The cost for virtualization is in buying really big hardware, oodles of
memory and many many cores, that are capable of running multiple VMs,
and having that hardware configured for redundancy, high availability
and failover.
With an HPC cluster, you are typically buying hardware that is as
stripp
gavin you mentioned costs, those are only incurred with xen if you need the
extra features such as server migration and other features. also if you dont
need those extra features couldnt you just live with the free version of
xen.
On Wed, Jan 27, 2010 at 11:42 AM, Geoff Galitz wrote:
>
>
> I've
I've had the good fortune to be in the HPC and also HA business for a few
years (10 years for HPC but only about 4 for HA). Given the current
approach for virtualization I don't see that Xen or other virtualization
technologies are good for HPC environments if the performance is a paramount
conc
Is it just me, or does HPC clustering and virtualization fall on
opposite ends of the spectrum?
With virtualization, you are pooling many virtual OS/server instances on
high availablility hardware, sharing memory and cpu as demanded,
oversubscribing. What would be idle time on one server, is util
;Jonathan Aquilina"
Cc: "Beowulf Mailing List"
Subject: Re: [Beowulf] clustering using xen virtualized machines
You may want to look at this:
Building A Virtual Cluster with Xen
http://www.clustermonkey.net//content/view/139/33/
--
Doug
has anyone tried clustering using xen
On the AWS ec2 side, we've been performing a range of tests including
full genome sequencing pipelines across varying numbers of nodes and
storage. The biggest challenge to date has been IO, particularly if the
smaller image systems are used. Where jobs are highly cpu bound, little
network (or
HPCwire have a feature on HPC Cloud computing:
http://www.hpcwire.com/specialfeatures/cloud_computing/
The contents of this email are confidential and for the exclusive use of the
intended recipient. If you receive this email in error you should not copy it,
retransmit it, use it or disclose i
chris not only the vm being portable yes you would take a hit yet from my
research into xen it seems like the paid version of citrix xen server has
some other nice features such as migration to a back up machine in case of
hardware failure.
when you all say performance hit how much of a hit are we
One of the virtualization trends I do see in HPC/clustering is in the
area of packaging up entire scientific applications into their own
custom VMs which contain all the necessary libraries, software
dependencies etc.
There is a performance hit now and implementation is clunky but I can
see
Is it just me, or does HPC clustering and virtualization fall on
opposite ends of the spectrum?
depends on your definitions. virtualization certainly conflicts with
those aspects of HPC which require bare-metal performance. even if you
can reduce the overhead of virtualization, the question is
do you guys think that virtualized clustering is the future?
___
Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
Ashley Pittman wrote:
On 25 Jan 2010, at 15:28, Jonathan Aquilina wrote:
has anyone tried clustering using xen based vm's. what is everyones take on
that? its something that popped into my head while in my lectures today.
I've been using Amazon ec2 for clustering for months now, from
>
> Is it just me, or does HPC clustering and virtualization fall on
> opposite ends of the spectrum?
>
Gavin, not necessarily. You could have a cluster of HPC compute nodes
running a minimal base OS.
Then install specific virtual machines with different OS/software stacks
each time your run a j
john i thank you for the encouragement. its better then what i get form
certain people i deal with in ubuntu channels. you mention diskless booting
using tftp and pxe. the problem though arises when u have a certain number
of nodes accessing the same disk simultaneously where disk I/O shoots
throug
for starters to save on resourses why not cut out the gui and go commandline to
free up some more of the shared resources, and 2ndly wouldnt offloading data
storage to a san or nfs storage server mitigate the disk I/O issues?
i honestly dont know much about xen as i just got my hands dirty with
On Tue, 2010-01-26 at 13:24 +, Tim Cutts wrote:
> 1) Applications with I/O patterns of large numbers of small disk
> operations are particularly painful (such as our ganglia server with
> all its thousands of tiny updates to RRD files). We've mitigated this
> by configuring Linux on th
for starters to save on resourses why not cut out the gui and go commandline
to free up some more of the shared resources, and 2ndly wouldnt offloading
data storage to a san or nfs storage server mitigate the disk I/O issues?
i honestly dont know much about xen as i just got my hands dirty with it
On 26 Jan 2010, at 1:24 pm, Tim Cutts wrote:
2) Raw device maps (where you pass a LUN straight through to a
single virtual machine, rather than carving the disk out of a
datastore) reduce contention and increase performance somewhat, at
the cost of using up device minor numbers on ESX qui
On 26 Jan 2010, at 12:00 pm, Jonathan Aquilina wrote:
does anyone have any benchmarks for I/O in a virtualized cluster?
I don't have formal benchmarks, but I can tell you what I see on my
VMware virtual machines in general:
Network I/O is reasonably fast - there's some additional latency,
does anyone have any benchmarks for I/O in a virtualized cluster?
___
Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
woudl somethign like this be useful for lets say setting up a system that
works with AI and voice recognition?
--
Jonathan Aquilina
___
Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode o
On 25 Jan 2010, at 15:28, Jonathan Aquilina wrote:
> has anyone tried clustering using xen based vm's. what is everyones take on
> that? its something that popped into my head while in my lectures today.
I've been using Amazon ec2 for clustering for months now, from a software
perspective it's
You may want to look at this:
Building A Virtual Cluster with Xen
http://www.clustermonkey.net//content/view/139/33/
--
Doug
> has anyone tried clustering using xen based vm's. what is everyones take
> on
> that? its something that popped into my head while in my lectures today.
>
> --
> Jon
has anyone tried clustering using xen based vm's. what is everyones take on
that? its something that popped into my head while in my lectures today.
--
Jonathan Aquilina
___
Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing
To cha
37 matches
Mail list logo
