Re: [Beowulf] copying big files (Henning Fehrmann)

Hi Perry E. Metzger wrote: > Is there a reason bittorrent isn't suited to this application? > Our investigations so far showed that bittorrent is only good if the files to be transferred fit well into main memory. If you exceed about 90-95% of the RAM your disks will be accessed a lot and the p

Re: [Beowulf] Weird CentOS Install Problem

Matt Lawrence wrote: On Fri, 8 Aug 2008, Jon Forrest wrote: What's weird about this is that the root file system starts on cylinder 1, as confirmed by the fdisk command. This is using a brand new SuperMicro X7SBE motherboard with the newest BIOS. I suggest you create a /boot partition of abou

Re: [Beowulf] Building new cluster - estimate

Matt Lawrence wrote: I really would like to have it all in one package. It's too easy to get things out of sync when doing changes by hand. I was figuring to find the appropriate sections of the CentOS spec file and use them. As always, other suggestions are welcome. You can add this in t

Re: [Beowulf] Weird CentOS Install Problem

On Fri, 8 Aug 2008, Jon Forrest wrote: I thought maybe some of you cluster people might have seen this. I have a brand new machine that will be the frontend of a cluster. It has a 3ware 9650 12-port RAID controller with 12 1TB drives attached. I used the "Boot Volume" feature in the RAID contr

[Beowulf] Weird CentOS Install Problem

I thought maybe some of you cluster people might have seen this. I have a brand new machine that will be the frontend of a cluster. It has a 3ware 9650 12-port RAID controller with 12 1TB drives attached. I used the "Boot Volume" feature in the RAID controller to make an 80GB boot volume. I inst

Re: [Beowulf] Re: computer Go

2008/8/8 Peter St. John <[EMAIL PROTECTED]> > The program was MoGo, > http://www.lri.fr/~gelly/MoGo.htm, > but I don't know anything about the "borrowed" hardware. > Peter > ... > > > I haven't found specifics for the machine or the team yet, but to quote t

Re: [Beowulf] LAM w/HPC Challenge: ld.so interface ?

Hi, Am 08.08.2008 um 20:47 schrieb Mikhail Kuzminsky: For some reasons I need to run HPC Challenge tests w/LAM MPI instead of OpenMPI. LAM was installed, and the classical test w/hello - using gcc - was performed successfully. But when I build hpcc executable, I see linker messages about

Re: [Beowulf] LAM w/HPC Challenge: ld.so interface ?

Sorry, how to find - which library contains this dl* modules ? I think you just need to link with -ldl ___ Beowulf mailing list, Beowulf@beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo

Re: [Beowulf] gcc 4.3 and acml 4.1.0 do not play together

Mark Kosmowski wrote: After a bit of struggling I found a post at the AMD Developer Forum stating that gcc 4.3 and acml 4.1.0 are not compatible. I've been reading a bunch of folks here using gcc 4.3 and just wanted to make everyone aware, hopefully prevent some time lost to futility. Have a gr

[Beowulf] LAM w/HPC Challenge: ld.so interface ?

For some reasons I need to run HPC Challenge tests w/LAM MPI instead of OpenMPI. LAM was installed, and the classical test w/hello - using gcc - was performed successfully. But when I build hpcc executable, I see linker messages about undefined references to dlsym/dlclose/dlopen/dlerror mod

RE: [Beowulf] copying big files (Henning Fehrmann)

> I will say that my dream would be for something like dolly to get some sort > of transfer recovery mechanism, though I realize that would be quite > difficult in such a topology. nettee has some failover and continuation capabilities at different points - but not what I think you want. The deve

Re: [Beowulf] copying big files (Henning Fehrmann)

"David Mathog" <[EMAIL PROTECTED]> writes: > Perry E. Metzger wrote: >> Is there a reason bittorrent isn't suited to this application? > > That would probably work too. > > I also recall reading about "broadcast" methods for doing this sort > of distribution. John Ioannidis designed a protocol

Re: [Beowulf] copying big files

stephen mulcahy <[EMAIL PROTECTED]> writes: > Perry E. Metzger wrote: >> bittorrent works quite well, and is trivial to use. It will use all >> your IO bandwidth between all nodes (which is what you want I >> presume.) > > What is the simplest app for setting up this kind of local torrenting > set

Re: [Beowulf] copying big files (Henning Fehrmann)

Perry E. Metzger wrote: > Is there a reason bittorrent isn't suited to this application? That would probably work too. I also recall reading about "broadcast" methods for doing this sort of distribution. I originally wrote nettee, which is a derivative of "dolly", because Ghost (which uses a b

RE: [Beowulf] copying big files (Henning Fehrmann)

I use dolly (http://www.cs.inf.ethz.ch/CoPs/patagonia/ and search for dolly) from which nettee is forked and pdsh (http://sourceforge.net/projects/pdsh). Both are great but dolly has certain advantages for my environment. In my case, I wrap it up in a service delivery tool for a 50 node cluste

Re: [Beowulf] copying big files

On Fri, 2008-08-08 at 11:59 -0400, Joe Landman wrote: > > The transmission of a single package is fast. The time of passing a > > particular > > package through the whole chain of nodes is short compared with time of the > > entire copying process. E.g., using jumbo frames a package can have the

Re: [Beowulf] copying big files

Perry E. Metzger wrote: bittorrent works quite well, and is trivial to use. It will use all your IO bandwidth between all nodes (which is what you want I presume.) What is the simplest app for setting up this kind of local torrenting setup? I've used public torrents before but assumed you ne

[Beowulf] Re: computer Go

The program was MoGo, http://www.lri.fr/~gelly/MoGo.htm, but I don't know anything about the "borrowed" hardware. Peter On 8/8/08, Peter St. John <[EMAIL PROTECTED]> wrote: > > The American Go Association (which has a free e-newsletter) at > http://www.usgo.org/ reports that a machine won an exhi

Re: [Beowulf] copying big files

Henning Fehrmann <[EMAIL PROTECTED]> writes: > Coping a big file onto all nodes in a cluster is a rather common problem. > I would have thought that there might be a standard tool for > distributing the files in an efficient way. So far, I haven't found one. bittorrent works quite well, and is t

Re: [Beowulf] copying big files (Henning Fehrmann)

"David Mathog" <[EMAIL PROTECTED]> writes: > The biggest problem with chain methods is that it is difficult to > recover if something breaks in the middle during the transfer. My > cluster is only 20 nodes and it has not been an issue, but on a 2000 > node cluster it probably would be. It is of

Re: [Beowulf] Building new cluster - estimate

Robert Latham <[EMAIL PROTECTED]> writes: > On Wed, Aug 06, 2008 at 09:41:35AM -0400, Perry E. Metzger wrote: >> >> Matt Lawrence <[EMAIL PROTECTED]> writes: >> > Could be. Given the long and sordid history of NFS, I prefer to not >> > use it whenever there are practical alternatives. >> >> NFS

[Beowulf] gcc 4.3 and acml 4.1.0 do not play together

After a bit of struggling I found a post at the AMD Developer Forum stating that gcc 4.3 and acml 4.1.0 are not compatible. I've been reading a bunch of folks here using gcc 4.3 and just wanted to make everyone aware, hopefully prevent some time lost to futility. Have a great weekend! Mark E. Ko

RE: [Beowulf] copying big files

> Has somebody written such a tool? eXludus have a product called Replicator http://www.exludus.com/ We have benchmarked an early version of it and it was pretty good at copying large files from an NFS server over gigabit to compute nodes. ___ Beowulf

[Beowulf] copying big files (Henning Fehrmann)

Henning Fehrmann <[EMAIL PROTECTED]> wrote: > Coping a big file onto all nodes in a cluster is a rather > common problem. I would have thought that there might be a > standard tool for distributing the files in an efficient way. > So far, I haven't found one. This is what I use: http://saf.bi

Re: [Beowulf] copying big files

Henning Fehrmann wrote: Hi everybody, Coping a big file onto all nodes in a cluster is a rather common problem. I would have thought that there might be a standard tool for distributing the files in an efficient way. So far, I haven't found one. Assuming one has a network design which allows

Re: [Beowulf] copying big files

Hallo Henning, Freitag, 8. August 2008, meintest Du: HF> Hi everybody, HF> One needs basically a daemon which handles copying requests and establishes HF> the connection to next node in the chain. Why a daemon? Just MPI that starts up the processes on the remote nodes during programm startup.

[Beowulf] copying big files

Hi everybody, Coping a big file onto all nodes in a cluster is a rather common problem. I would have thought that there might be a standard tool for distributing the files in an efficient way. So far, I haven't found one. Assuming one has a network design which allows non blocking full duplex wi

Re: [Beowulf] Building new cluster - estimate

On Wed, Aug 06, 2008 at 09:41:35AM -0400, Perry E. Metzger wrote: > > Matt Lawrence <[EMAIL PROTECTED]> writes: > > Could be. Given the long and sordid history of NFS, I prefer to not > > use it whenever there are practical alternatives. > > NFS is a fine protocol and works very well. However, t

[Beowulf] computer Go

The American Go Association (which has a free e-newsletter) at http://www.usgo.org/ reports that a machine won an exhibition game with a master last night at the US Go Congress. This isn't really historic; the master, Myungwan Kim, is an 8 dan professional, and gave 9 handicap stones to the machin