Re: [Beowulf] ARM cpu's and development boards and research

2012-11-27 Thread Mark Hahn
> What Bill has just described is known as an "Amdahl-balanced system", > and is the design philosophy between the IBM Blue Genes and also > SiCortex. In my opinion, this is the future of HPC. Use lower power, > slower processors, and then try to improve network performance to reduce > the cost of

Re: [Beowulf] hadoop

2012-11-27 Thread Prentice Bisbal
On 11/27/2012 10:39 AM, Massimiliano Fatica wrote: > There is a Tegra 3 + GPU development platform, called CARMA. > It has an on-board SATA controller too. > > CARMA = Cuda on ARM Architecture. Never heard of it before SC12, but heard the term plenty once I got there. Potentially useful link bel

Re: [Beowulf] ARM cpu's and development boards and research

2012-11-27 Thread Vincent Diepeveen
On Nov 28, 2012, at 12:17 AM, Prentice Bisbal wrote: > > On 11/27/2012 03:37 PM, Douglas Eadline wrote: >> >>> My interest in Arm has been the flip side of balancing flops to >>> network >>> bandwidth. A standard dual socket (AMD or Intel) can trivially >>> saturate >>> GigE. One option for

Re: [Beowulf] ARM cpu's and development boards and research

2012-11-27 Thread Prentice Bisbal
On 11/27/2012 03:37 PM, Douglas Eadline wrote: > >> My interest in Arm has been the flip side of balancing flops to network >> bandwidth. A standard dual socket (AMD or Intel) can trivially saturate >> GigE. One option for improving the flops/network balance is to add >> network bandwidth with I

Re: [Beowulf] ARM cpu's and development boards and research

2012-11-27 Thread Prentice Bisbal
On 11/27/2012 03:39 PM, Vincent Diepeveen wrote: > On Nov 27, 2012, at 9:05 PM, Bill Broadley wrote: > >> On 11/27/2012 07:46 AM, Vincent Diepeveen wrote: >>> i dug around in price of ARMs and development boards. >>> >>> If you just buy a handful most interesting offer seems to be >>> >>> http://

Re: [Beowulf] hadoop

2012-11-27 Thread Prentice Bisbal
On 11/27/2012 07:21 AM, Jonathan Aquilina wrote: > Vincent I dont have anything setup just hatching crazy ideas which atm > i dont have time to move forward with anything :( Hatching crazy ideas? That's Vincent's job. -- Prentice ___ Beowulf mailing li

Re: [Beowulf] ARM cpu's and development boards and research

2012-11-27 Thread Douglas Eadline
> > My interest in Arm has been the flip side of balancing flops to network > bandwidth. A standard dual socket (AMD or Intel) can trivially saturate > GigE. One option for improving the flops/network balance is to add > network bandwidth with Infiniband. Another is a slower, cheaper, cooler >

Re: [Beowulf] ARM cpu's and development boards and research

2012-11-27 Thread Vincent Diepeveen
On Nov 27, 2012, at 9:05 PM, Bill Broadley wrote: > On 11/27/2012 07:46 AM, Vincent Diepeveen wrote: >> i dug around in price of ARMs and development boards. >> >> If you just buy a handful most interesting offer seems to be >> >> http://www.hardkernel.com/renewal_2011/products/prdt_info.php? >>

Re: [Beowulf] ARM cpu's and development boards and research

2012-11-27 Thread Bill Broadley
On 11/27/2012 07:46 AM, Vincent Diepeveen wrote: > i dug around in price of ARMs and development boards. > > If you just buy a handful most interesting offer seems to be > > http://www.hardkernel.com/renewal_2011/products/prdt_info.php? > g_code=G133999328931 > > it's $129 and has a quad core A

Re: [Beowulf] Fwd: warewulf - cannot log into nodes

2012-11-27 Thread Gus Correa
On 11/27/2012 02:14 AM, Duke Nguyen wrote: > On 11/27/12 1:44 PM, Christopher Samuel wrote: >> -BEGIN PGP SIGNED MESSAGE- >> Hash: SHA1 >> >> On 27/11/12 15:51, Duke Nguyen wrote: >> >>> Thanks! Yes, I am trying to get the system work with >>> Torque/Maui/OpenMPI now. >> Make sure you build

Re: [Beowulf] Fwd: warewulf - cannot log into nodes

2012-11-27 Thread Gus Correa
On 11/27/2012 01:52 PM, Gus Correa wrote: > On 11/27/2012 02:14 AM, Duke Nguyen wrote: >> On 11/27/12 1:44 PM, Christopher Samuel wrote: >>> -BEGIN PGP SIGNED MESSAGE- >>> Hash: SHA1 >>> >>> On 27/11/12 15:51, Duke Nguyen wrote: >>> Thanks! Yes, I am trying to get the system work with

Re: [Beowulf] hadoop

2012-11-27 Thread Eugen Leitl
On Tue, Nov 27, 2012 at 11:50:00AM -0500, Ellis H. Wilson III wrote: > > Not at all. This particular application is to derive optimal > > feature extraction algorithms from high-resolution volumetric data > > (mammal or primate connectome). At ~8 nm, even a mouse will > > produce a mountain of str

Re: [Beowulf] hadoop

2012-11-27 Thread Ellis H. Wilson III
On 11/27/2012 11:34 AM, Eugen Leitl wrote: > On Tue, Nov 27, 2012 at 11:13:25AM -0500, Ellis H. Wilson III wrote: > >> Are these problems EP such that they could be entirely Map tasks? > > Not at all. This particular application is to derive optimal > feature extraction algorithms from high-resolut

Re: [Beowulf] hadoop

2012-11-27 Thread Eugen Leitl
On Tue, Nov 27, 2012 at 11:13:25AM -0500, Ellis H. Wilson III wrote: > Are these problems EP such that they could be entirely Map tasks? Not at all. This particular application is to derive optimal feature extraction algorithms from high-resolution volumetric data (mammal or primate connectome).

Re: [Beowulf] hadoop

2012-11-27 Thread Ellis H. Wilson III
On 11/27/2012 08:59 AM, Eugen Leitl wrote: > On Tue, Nov 27, 2012 at 09:10:32AM +0100, Jonathan Aquilina wrote: >> Hey guys I was looking at the hadoop page and it got me wondering. is it >> possible to cluster together storage servers? If so how efficient would a >> cluster of them be? > > An inte

Re: [Beowulf] hadoop

2012-11-27 Thread Ellis H. Wilson III
On 11/27/2012 09:54 AM, Douglas Eadline wrote: > >> not as efficient as gluster I would venture. > > OrangeFS (PVFS2) has been shown to work as good as or better than > HDFS I'm not going to take a side on this one since they are in many respects solving different problems in different domains, b

[Beowulf] ARM cpu's and development boards and research

2012-11-27 Thread Vincent Diepeveen
i dug around in price of ARMs and development boards. If you just buy a handful most interesting offer seems to be http://www.hardkernel.com/renewal_2011/products/prdt_info.php? g_code=G133999328931 it's $129 and has a quad core ARM cortex A9 at 1.4Ghz and a mali gpu on it. So from raw cpu c

Re: [Beowulf] hadoop

2012-11-27 Thread Massimiliano Fatica
There is a Tegra 3 + GPU development platform, called CARMA. It has an on-board SATA controller too. Massimiliano On Tue, Nov 27, 2012 at 7:28 AM, Eugen Leitl wrote: > On Tue, Nov 27, 2012 at 04:14:06PM +0100, Bogdan Costescu wrote: > > On Tue, Nov 27, 2012 at 3:40 PM, Eugen Leitl wrote: > >

Re: [Beowulf] hadoop

2012-11-27 Thread Eugen Leitl
On Tue, Nov 27, 2012 at 04:14:06PM +0100, Bogdan Costescu wrote: > On Tue, Nov 27, 2012 at 3:40 PM, Eugen Leitl wrote: > > Please do not top-post and please trim your replies. > > Thanks for the reminder ! It bugs me too... > > > I would be > > looking at least for an Exynos 5... > > ... to proc

Re: [Beowulf] hadoop

2012-11-27 Thread Jonathan Aquilina
> I'm still not convinced about computational capabilities of ARM cores. > So how about letting the ARM core deal with I/O and adding a dedicated > computing unit in form of FPGA ? Not suggesting a (GP)GPU to keep in > line with the low power envelope. > > What im interested in is seeing how well a

Re: [Beowulf] hadoop

2012-11-27 Thread Bogdan Costescu
On Tue, Nov 27, 2012 at 3:40 PM, Eugen Leitl wrote: > Please do not top-post and please trim your replies. Thanks for the reminder ! It bugs me too... > I would be > looking at least for an Exynos 5... > ... to process code locally ... I'm still not convinced about computational capabilities of

Re: [Beowulf] hadoop

2012-11-27 Thread Douglas Eadline
> not as efficient as gluster I would venture. OrangeFS (PVFS2) has been shown to work as good as or better than HDFS -- Doug > > > 2012/11/27 Jonathan Aquilina > >> Hey guys I was looking at the hadoop page and it got me wondering. is it >> possible to cluster together storage servers? If s

Re: [Beowulf] hadoop

2012-11-27 Thread Eugen Leitl
On Tue, Nov 27, 2012 at 03:01:57PM +0100, Jonathan Aquilina wrote: > One thing that really caught my eye was seamicros new storage servers up to > 5 petabytes of storage Please do not top-post and please trim your replies. Message below unchanged for illustration. The Seamicro is not the same thi

Re: [Beowulf] hadoop

2012-11-27 Thread Jonathan Aquilina
One thing that really caught my eye was seamicros new storage servers up to 5 petabytes of storage On Tue, Nov 27, 2012 at 2:59 PM, Eugen Leitl wrote: > On Tue, Nov 27, 2012 at 09:10:32AM +0100, Jonathan Aquilina wrote: > > Hey guys I was looking at the hadoop page and it got me wondering. is it

Re: [Beowulf] hadoop

2012-11-27 Thread Eugen Leitl
On Tue, Nov 27, 2012 at 09:10:32AM +0100, Jonathan Aquilina wrote: > Hey guys I was looking at the hadoop page and it got me wondering. is it > possible to cluster together storage servers? If so how efficient would a > cluster of them be? An interesting problem would be to use reasonably powerful

Re: [Beowulf] hadoop

2012-11-27 Thread Peter Clapham
Lo all, I've pinged Ellis a mail with a couple of specific Q's but I wondered if anyone here has any experience with disco (http://discoproject.org/). I'm still in the early days of testing but would be interested to hear if anyone has an installation running at scale or in anger. Thanks Pete -

Re: [Beowulf] hadoop

2012-11-27 Thread Jonathan Aquilina
> I remember he was 10 years ago living at Cyprus (if i remember well) > busy exploring the similar things in linux :) > Malta actually Vincent. Right now I am just toying with some ideas as I have a business concept I would like to move forward with once i finish my studies in may as mentioned i

Re: [Beowulf] hadoop

2012-11-27 Thread Vincent Diepeveen
Trying to simulate a beowulf cluster at a laptop with everything that belongs to it, is pretty much nonsense. No hard feelings towards Jonathan there, he's excused. I remember he was 10 years ago living at Cyprus (if i remember well) busy exploring the similar things in linux :) Speaking of

Re: [Beowulf] hadoop

2012-11-27 Thread Ellis H. Wilson III
On 11/27/2012 08:14 AM, Vincent Diepeveen wrote: > Don't post something ridicioulous like that. You both are right, so lets stop antagonizing the antagonist here. Laptops are a reasonable place to toy around with and educate oneself about Hadoop, but they are also not (obviously, I don't think t

[Beowulf] Spanner: synchronizing the largest global database

2012-11-27 Thread Eugen Leitl
http://www.wired.com/wiredenterprise/2012/11/google-spanner-time/all/ (it's about time) Exclusive: Inside Google Spanner, the Largest Single Database on Earth By Cade Metz 11.26.12 6:30 AM Each morning, when Andrew Fikes sat down at his desk inside Google headquarters in Mountain View, Calif

Re: [Beowulf] hadoop

2012-11-27 Thread Peter Clapham
On 27/11/2012 13:14, Vincent Diepeveen wrote: > Don't post something ridicioulous like that. > > That 64MB they use of course as with infiniband shipping a message of > 64MB wil get you the full bandwidth over the network > and keep number of messages down and infiniband doesn't work at your >

Re: [Beowulf] hadoop

2012-11-27 Thread Jonathan Aquilina
Wasnt intending to open up a can of worms here, its just me thinking out side the box and coming up with crazy and far fetched ideas On Tue, Nov 27, 2012 at 2:14 PM, Vincent Diepeveen wrote: > Don't post something ridicioulous like that. > > That 64MB they use of course as with infiniband shippi

Re: [Beowulf] hadoop

2012-11-27 Thread Vincent Diepeveen
Don't post something ridicioulous like that. That 64MB they use of course as with infiniband shipping a message of 64MB wil get you the full bandwidth over the network and keep number of messages down and infiniband doesn't work at your laptop. On Nov 27, 2012, at 1:24 PM, Andrew Holway wrote

Re: [Beowulf] hadoop

2012-11-27 Thread Jonathan Aquilina
All I am going to say is I have tons upon tons of learning to do in regards to linux. Currently learning to setup a multiple domain setup for emails. On Tue, Nov 27, 2012 at 1:24 PM, Andrew Holway wrote: > Laptops are perfectly good for functionality testing and learning. Get > back in your box V

Re: [Beowulf] hadoop

2012-11-27 Thread Andrew Holway
Laptops are perfectly good for functionality testing and learning. Get back in your box Vincent. 2012/11/27 Jonathan Aquilina > Vincent I dont have anything setup just hatching crazy ideas which atm i > dont have time to move forward with anything :( > > > On Tue, Nov 27, 2012 at 1:18 PM, Vince

Re: [Beowulf] hadoop

2012-11-27 Thread Jonathan Aquilina
Vincent I dont have anything setup just hatching crazy ideas which atm i dont have time to move forward with anything :( On Tue, Nov 27, 2012 at 1:18 PM, Vincent Diepeveen wrote: > > On Nov 27, 2012, at 10:19 AM, Jonathan Aquilina wrote: > > > Sadly I have no experience in a cluster environment

Re: [Beowulf] hadoop

2012-11-27 Thread Vincent Diepeveen
On Nov 27, 2012, at 10:19 AM, Jonathan Aquilina wrote: > Sadly I have no experience in a cluster environment :( I do have my > old laptop and hopefully have a 2nd one soon. Is it possible to > have a mixed cluster with gluster or hadoop or any other clustering > technology which is a hetrog

Re: [Beowulf] hadoop

2012-11-27 Thread Jonathan Aquilina
Basically my idea is to setup a green self sustaining data center. My hang up is finding investors. Where I am electricity is super expensive and the costs are passed onto the clients hosting with the current Local data centers. Would be great if there were some investors on this list, but I think

Re: [Beowulf] hadoop

2012-11-27 Thread Andrew Holway
2012/11/27 Jonathan Aquilina > Interesting indeed. Does LVM span across multiple storage servers? There is Clustered LVM but I dont think this is what your looking for. CLVM allows you to have a shared storage target such as an iSCSI box and give one LV to one box and another LV to another box

Re: [Beowulf] hadoop

2012-11-27 Thread Jonathan Aquilina
Interesting indeed. Does LVM span across multiple storage servers? That is another issue and im not trying to open up a can of worms but what is the advantage of using KVM over xen or even citrix xen server On Tue, Nov 27, 2012 at 10:32 AM, Andrew Holway wrote: > Sure. With virtualisation by KVM

Re: [Beowulf] Fwd: warewulf - cannot log into nodes

2012-11-27 Thread Duke Nguyen
On 11/27/12 2:14 PM, Duke Nguyen wrote: > On 11/27/12 1:44 PM, Christopher Samuel wrote: >> -BEGIN PGP SIGNED MESSAGE- >> Hash: SHA1 >> >> On 27/11/12 15:51, Duke Nguyen wrote: >> >>> Thanks! Yes, I am trying to get the system work with >>> Torque/Maui/OpenMPI now. >> Make sure you build Op

Re: [Beowulf] hadoop

2012-11-27 Thread Andrew Holway
Sure. With virtualisation by KVM. (although not sure about the windows and mac) Maybe VirtualBox? I'm currently designing a cloud computing platform that uses NFS for the worker virtual machines (databases and webservers etc) and the local storage can be exported using gluster or whatever. Say if

Re: [Beowulf] hadoop

2012-11-27 Thread Jonathan Aquilina
Sadly I have no experience in a cluster environment :( I do have my old laptop and hopefully have a 2nd one soon. Is it possible to have a mixed cluster with gluster or hadoop or any other clustering technology which is a hetrogenous cluster in terms of linux machines and mac machines and if someon

Re: [Beowulf] hadoop

2012-11-27 Thread Andrew Holway
not as efficient as gluster I would venture. 2012/11/27 Jonathan Aquilina > Hey guys I was looking at the hadoop page and it got me wondering. is it > possible to cluster together storage servers? If so how efficient would a > cluster of them be? > > -- > Jonathan Aquilina > > _

[Beowulf] hadoop

2012-11-27 Thread Jonathan Aquilina
Hey guys I was looking at the hadoop page and it got me wondering. is it possible to cluster together storage servers? If so how efficient would a cluster of them be? -- Jonathan Aquilina ___ Beowulf mailing list, Beowulf@beowulf.org sponsored by Pengui