Re: [Beowulf] single machine with 500 GB of RAM

2013-01-09 Thread Mark Hahn
> procs. Within each process the accesses to their "cube" of data were > near to completely random. "completely random" is a bit like von Neumann's "state of sin" ;) if they managed to make actually uniform random accesses, they'd have discovered a new PRNG, possibly the most conpute-intensive k

Re: [Beowulf] single machine with 500 GB of RAM

2013-01-09 Thread Ellis H. Wilson III
On 01/09/2013 08:06 PM, Vincent Diepeveen wrote: > > On Jan 10, 2013, at 1:47 AM, Ellis H. Wilson III wrote: >> [snip] > >> My attention to them has revolved more around the latency issues that >> arise when you add a HW raid controller and the SATA protocol between >> the CPU and the SSD. This is

Re: [Beowulf] single machine with 500 GB of RAM

2013-01-09 Thread Vincent Diepeveen
On Jan 10, 2013, at 1:47 AM, Ellis H. Wilson III wrote: > [snip] > My attention to them has revolved more around the latency issues that > arise when you add a HW raid controller and the SATA protocol between > the CPU and the SSD. This is exacerbated if the access patterns do > not > remain c

Re: [Beowulf] single machine with 500 GB of RAM

2013-01-09 Thread Ellis H. Wilson III
On 01/09/2013 05:53 PM, Mark Hahn wrote: > given the "slicing" methodology that this application uses for > decomposition, I wonder whether it's actually closer to sequential > in its access patterns, rather than random. the point here is that > you absolutely must have ram if your access is rando

Re: [Beowulf] BIG 'ram' using SSDs - was single machine with 500 GB of RAM

2013-01-09 Thread Stuart Barkley
On Wed, 9 Jan 2013 at 08:27 -, Vincent Diepeveen wrote: > What would be a rather interesting thought for building a single box > dirt cheap with huge 'RAM' is the idea of having 1 fast RAID array > of SSD's function as the 'RAM'. We recently had a chance to look at something like this at a sm

Re: [Beowulf] K Computer Architecture

2013-01-09 Thread Rayson Ho
Hi Prentice, If you are *still* interested in the K computer, see Vol.48 of the Fujitsu Global Journal: http://www.fujitsu.com/global/news/publications/periodicals/fstj/archives/vol48-3.html It discloses many internal workings of the K computer - Open MPI on K (ToFu interconnect), SPARC64 VIIIfx

Re: [Beowulf] single machine with 500 GB of RAM

2013-01-09 Thread Mark Hahn
> However, looking at the user manual for this application however, I > suspect the bulk of the work can be made parallel, in contrast to the > original post: yes - the very first page of preface mentions a straightforward decomposition that scales to 40 processors. that would be "shared-nothin

Re: [Beowulf] Configuration management tools/strategy

2013-01-09 Thread Bill Broadley
On 01/06/2013 05:38 AM, Walid wrote: > Dear All, > > At work we are starting to evaluate Configuration management to be used > to manage several diverse hpc clusters We currently managing 15 clusters with puppet and am very pleased with puppet. Puppet is one of the critical pieces that allows us

Re: [Beowulf] BIG 'ram' using SSDs - was single machine with 500 GB of RAM

2013-01-09 Thread Lux, Jim (337C)
We do a fair amount of "giant capture" or "giant playback" kinds of applications with very large buffers, which is sort of the same kind of thing. It's a pain, and it usually requires special purpose hardware of one sort or another. Sure, it's "off the shelf", but it's small volume and expensive,

Re: [Beowulf] BIG 'ram' using SSDs - was single machine with 500 GB of RAM

2013-01-09 Thread Ellis H. Wilson III
On 01/09/2013 02:24 PM, Vincent Diepeveen wrote: > > On Jan 9, 2013, at 8:02 PM, Ellis H. Wilson III wrote: >> >> Yea, it's computer science, and I'd love to see you try to toss 16 >> crappy SSDs in a box with a crappy RAID controller and get this easy >> 2.7GB/s random accesses you are touting. N

Re: [Beowulf] BIG 'ram' using SSDs - was single machine with 500 GB of RAM

2013-01-09 Thread Vincent Diepeveen
On Jan 9, 2013, at 8:02 PM, Ellis H. Wilson III wrote: > > Yea, it's computer science, and I'd love to see you try to toss 16 > crappy SSDs in a box with a crappy RAID controller and get this easy > 2.7GB/s random accesses you are touting. Not going to happen. > If you had done effort reading wh

Re: [Beowulf] single machine with 500 GB of RAM

2013-01-09 Thread Gus Correa
On 01/09/2013 12:00 PM, Andrew Holway wrote: > As its a single thread I doubt that faster memory is going to help you much. > It's going to suck whatever you do. > Hi I don't know anything about computational chemistry or its grid/mesh requirements, but if you look at the makefile, you'll see t

Re: [Beowulf] single machine with 500 GB of RAM

2013-01-09 Thread Andrew Holway
> I would think faster memory would be the only thing that could be done > about it, Indeed but I am imagining that the law of diminishing returns is going to kick in here hard and fast. ___ Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin

Re: [Beowulf] BIG 'ram' using SSDs - was single machine with 500 GB of RAM

2013-01-09 Thread Ellis H. Wilson III
On 01/09/2013 01:21 PM, Vincent Diepeveen wrote: > On Jan 9, 2013, at 4:33 PM, Ellis H. Wilson III wrote: >> On 01/09/2013 08:27 AM, Vincent Diepeveen wrote: >>> What would be a rather interesting thought for building a single box >>> dirt cheap with huge 'RAM' >>> is the idea of having 1 fast RAID

Re: [Beowulf] single machine with 500 GB of RAM

2013-01-09 Thread Vincent Diepeveen
On Jan 9, 2013, at 6:00 PM, Andrew Holway wrote: > As its a single thread I doubt that faster memory is going to help > you much. It's going to suck whatever you do. Memory latency of DDR3 is significantly faster than DDR2 for random accesses. And i don't speak about 32 bytes only (as DDR3

Re: [Beowulf] single machine with 500 GB of RAM

2013-01-09 Thread Vincent Diepeveen
On Jan 9, 2013, at 5:29 PM, Jörg Saßmannshausen wrote: > Dear all, > > many thanks for the quick reply and all the suggestions. > > The code we want to use is that one here: > > http://www.cpfs.mpg.de/~kohout/dgrid.html > > Feel free to download and dig into the code. I am no expert in > Fortra

Re: [Beowulf] single machine with 500 GB of RAM

2013-01-09 Thread Ellis H. Wilson III
On 01/09/2013 12:00 PM, Andrew Holway wrote: > As its a single thread I doubt that faster memory is going to help you much. > It's going to suck whatever you do. I would think faster memory would be the only thing that could be done about it, presuming it's single threaded for very good reason (

Re: [Beowulf] BIG 'ram' using SSDs - was single machine with 500 GB of RAM

2013-01-09 Thread Vincent Diepeveen
On Jan 9, 2013, at 4:33 PM, Ellis H. Wilson III wrote: > On 01/09/2013 08:27 AM, Vincent Diepeveen wrote: >> What would be a rather interesting thought for building a single box >> dirt cheap with huge 'RAM' >> is the idea of having 1 fast RAID array of SSD's function as the >> 'RAM'. > > This

Re: [Beowulf] single machine with 500 GB of RAM

2013-01-09 Thread Joe Landman
On 1/9/13 11:29 AM, Jörg Saßmannshausen wrote: > I am wondering about the vSMP / ScaleMP suggestion from Joe. If I am using an > InfiniBand network here, would I be able to spread the 'bottlenecks' a bit > better? What I am after is, when I tested out the InfiniBand on the new > cluster Well, it

Re: [Beowulf] single machine with 500 GB of RAM

2013-01-09 Thread Andrew Holway
As its a single thread I doubt that faster memory is going to help you much. It's going to suck whatever you do. Am 9 Jan 2013 um 17:29 schrieb Jörg Saßmannshausen : > Dear all, > > many thanks for the quick reply and all the suggestions. > > The code we want to use is that one here: > > htt

Re: [Beowulf] single machine with 500 GB of RAM

2013-01-09 Thread Jörg Saßmannshausen
Dear all, many thanks for the quick reply and all the suggestions. The code we want to use is that one here: http://www.cpfs.mpg.de/~kohout/dgrid.html Feel free to download and dig into the code. I am no expert in Fortran so I won't be able to help you much if you got specific questions to the

Re: [Beowulf] single machine with 500 GB of RAM

2013-01-09 Thread Sabuj Pattanayek
Well, the HP ProLiant DL580 G7 has 64 DIMM slots (supporting 2TB) and supports up to 4 E5-4xxx processors, problem is it looks like they'll only sell you a minimum of 2 CPUs. On Wed, Jan 9, 2013 at 9:43 AM, Sabuj Pattanayek wrote: >> I am not aware of a single socket motherboard that can cope wit

Re: [Beowulf] single machine with 500 GB of RAM

2013-01-09 Thread Sabuj Pattanayek
> I am not aware of a single socket motherboard that can cope with 500GB > ram. 2 socket motherboards support about 256GB (128GB per processor) > or so at the moment and quad socket boards upto 1TB. For x86 architectures, the most I'm seeing in any available motherboard is 12 DIMMs per socket with

Re: [Beowulf] BIG 'ram' using SSDs - was single machine with 500 GB of RAM

2013-01-09 Thread Ellis H. Wilson III
On 01/09/2013 08:27 AM, Vincent Diepeveen wrote: > What would be a rather interesting thought for building a single box > dirt cheap with huge 'RAM' > is the idea of having 1 fast RAID array of SSD's function as the 'RAM'. This may be a more inexpensive route, but let's all note that the raw late

Re: [Beowulf] single machine with 500 GB of RAM

2013-01-09 Thread Andrew Holway
> So if I am using a single socket motherboard, would that not be faster or does > a single CPU not cope with that amount of memory? I am not aware of a single socket motherboard that can cope with 500GB ram. 2 socket motherboards support about 256GB (128GB per processor) or so at the moment and q

Re: [Beowulf] BIG 'ram' using SSDs - was single machine with 500 GB of RAM

2013-01-09 Thread atchley tds.net
There are papers that describe doing this. For example, see: http://www.csm.ornl.gov/~vazhkuda/NVMalloc.html On Wed, Jan 9, 2013 at 8:59 AM, John Hearns wrote: > I agree with Vincent. > > I have often thought that wr should be able to run codes which need high > memory sizes by using SSDs > >

Re: [Beowulf] BIG 'ram' using SSDs - was single machine with 500 GB of RAM

2013-01-09 Thread John Hearns
I agree with Vincent. I have often thought that wr should be able to run codes which need high memory sizes by using SSDs ___ Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe

Re: [Beowulf] single machine with 500 GB of RAM

2013-01-09 Thread Vincent Diepeveen
Take a good look at the code. You can probably get a better IPC out of it with a few cores by using a tricky form of parallellisation. Yet then it stops as far as parallellization is concerned :) On Jan 9, 2013, at 2:05 PM, Jörg Saßmannshausen wrote: > Hi John, > > I was toying with this idea

[Beowulf] BIG 'ram' using SSDs - was single machine with 500 GB of RAM

2013-01-09 Thread Vincent Diepeveen
What would be a rather interesting thought for building a single box dirt cheap with huge 'RAM' is the idea of having 1 fast RAID array of SSD's function as the 'RAM'. You can get a bandwidth of 2 GB/s then to the SSD's "RAM" pretty easily, yet for some calculations that bandwidth might be eno

Re: [Beowulf] single machine with 500 GB of RAM

2013-01-09 Thread Jörg Saßmannshausen
Hi John, I was toying with this idea as well. However, what I do not know is: as it is a single core job, do I really need the second CPU for the memory management or would that slow down things? In other words, does it look like that: Memoryrequest CPU#0 -> CPU#1 -> RAM So if I am using a singl

Re: [Beowulf] single machine with 500 GB of RAM

2013-01-09 Thread Joe Landman
On 01/09/2013 07:42 AM, Jörg Saßmannshausen wrote: > Dear all, > > Happy New Year! > > I was wondering whether people on the list here have some first hand > experiences with this. I have been asked to purchase a single machine with > around 500 GB of RAM. We would not need more than 8 cores here.

Re: [Beowulf] single machine with 500 GB of RAM

2013-01-09 Thread Vincent Diepeveen
(resend - i see the previous time i wrote the stuff below message was held for some reason @ beowulf) Hi, What budget do you have and how fast must latency be to the RAM? There will be a LOT of solutions to this. If you can get a box with way more dimm slots that's always the ideal solution

Re: [Beowulf] single machine with 500 GB of RAM

2013-01-09 Thread John Hearns
You will need to populate all of the CPU sockets if you are filing all the DIMM slots. So yoy probably need more than one CPU On Jan 9, 2013 12:42 PM, "Jörg Saßmannshausen" wrote: > > Dear all, > > Happy New Year! > > I was wondering whether people on the list here have some first hand > experienc

[Beowulf] single machine with 500 GB of RAM

2013-01-09 Thread Jörg Saßmannshausen
Dear all, Happy New Year! I was wondering whether people on the list here have some first hand experiences with this. I have been asked to purchase a single machine with around 500 GB of RAM. We would not need more than 8 cores here. The job simply needs that much of memory (and even then it i