Re: [Beowulf] Doing i/o at a small cluster

2012-08-18 Thread Andrew Holway
> So some sort of distributed file system seems the best option, and a > lot cheaper and a lot faster than a dedicated fileserver > that will not be able to keep up. a) ZFS doesn't use hardware raid. at all. ever. b) 500MB/s (actually 1GB/s) of I/O will chew up a quite large amount of resource. J

Re: [Beowulf] Doing i/o at a small cluster

2012-08-18 Thread Andrew Holway
2012/8/17 Vincent Diepeveen : > The homepage looks very commercial and they have a free trial on it. > You refer to the free trial? http://nexentastor.org/ - Sorry wrong link. Its a commercially backed open source project. > Means buy raid controller. That's extra cost. That depends upon what > i

Re: [Beowulf] Doing i/o at a small cluster

2012-08-18 Thread Vincent Diepeveen
On Aug 18, 2012, at 3:19 PM, Ellis H. Wilson III wrote: > On 08/17/2012 12:04 PM, Vincent Diepeveen wrote: >> Yes i realize that. In principle you're looking at a 1050 files or so >> that get effectively generated >> and to generate each file half a dozen of huge files get created. >> >> Now in t

Re: [Beowulf] Doing i/o at a small cluster

2012-08-18 Thread Vincent Diepeveen
On Aug 18, 2012, at 1:04 PM, Andrew Holway wrote: > 2012/8/17 Vincent Diepeveen : >> The homepage looks very commercial and they have a free trial on it. >> You refer to the free trial? > > http://nexentastor.org/ - Sorry wrong link. Its a commercially backed > open source project. > >> Means buy

Re: [Beowulf] Doing i/o at a small cluster

2012-08-18 Thread Ellis H. Wilson III
On 08/17/2012 12:04 PM, Vincent Diepeveen wrote: > Yes i realize that. In principle you're looking at a 1050 files or so > that get effectively generated > and to generate each file half a dozen of huge files get created. > > Now in theory generating them is embarrassingly parallel except that to >