Re: [Beowulf] strange problem with large file moving between server

2014-09-21 Thread Jörg Saßmannshausen
Hi Andrew, thanks. I will look into that. It is good to hear it is ready for production now. The last time I looked into it it was not. All the best Jörg On Sonntag 21 September 2014 you wrote: > > Regarding ZFS: is that available for Linux now? I lost a bit track here. > > Yes. > > http://

Re: [Beowulf] strange problem with large file moving between server

2014-09-21 Thread Andrew Holway
> > Regarding ZFS: is that available for Linux now? I lost a bit track here. > Yes. http://zfsonlinux.org/ I would say its ready for production now. Intel are about to start supporting it under Lustre in the next couple of months and they are typically careful about such things. Cheers, Andrew

Re: [Beowulf] strange problem with large file moving between server

2014-09-21 Thread Jörg Saßmannshausen
Hi Alex, thanks for the feedback. I still got the original data so that is not a problem right now. What worries me is even if I restore the data right now can I trust the system? It is a RAID5 I am using and the discs are new. I have formated the disc space on Thursday so the file system is ne

Re: [Beowulf] strange problem with large file moving between server

2014-09-21 Thread Alex Chekholko
Hi Jörg, Sounds like a "typical" but very uncommon silent data corruption problem. If you have another copy of the data, compare to that? If you don't have another copy, accept the fact that some of your data maybe got silently corrupted. Most RAID controllers do periodic "scrubbing"; was your In

[Beowulf] strange problem with large file moving between server

2014-09-21 Thread Jörg Saßmannshausen
Dear all, I got a rather strange problem with one of my file servers which I recently have upgraded in order to accommodate more disc space. The problem: I have copies the files from the old file space to a temporary disc storage space using this rsync command: rsync -vrltH -pgo --stats -D -