On 12 November 2012 11:26, Duke Nguyen <duke.li...@gmx.com> wrote: > On 11/12/12 4:42 PM, Tim Cutts wrote: >> On 12 Nov 2012, at 03:50, Duke Nguyen <duke.li...@gmx.com> wrote: >> >>> On 11/9/12 7:26 PM, Bogdan Costescu wrote: >>>> On Fri, Nov 9, 2012 at 7:19 AM, Christopher Samuel >>>> <sam...@unimelb.edu.au> wrote: >>>>> So JBODs with LVM on top and XFS on top of that could be resized on >>>>> the fly. You can do the same with ext[34] as well (from memory). >>> We also thought of using LVM on top of RAID disks, but never think of >>> XFS. Why do we need XFS and how does this compare with GPFS? >> XFS is not a parallel filesystem like GPFS, but a lot of us use it because >> it's so easy to grow the filesystem. I frequently do this with virtual >> machines to expand their storage. >> >> 1) Add new virtual disk >> 2) Scan SCSI buses in Linux >> 3) pvcreate the new device >> 4) vgextend >> 5) lvextend >> 6) xfs_growfs >> >> Job done. >> >> On systems in our data centre, we tend to use XFS for most local >> filesystems, unless we have a good reason to use something else. > > Is there any reason you do not use GlusterFS on top of XFS? Can I do > that? With that, can I have a parallel system similar to GPFS?
Not quite. GPFS (and RedHat's GFS) mediate parallel access to the block devices. So, more than one host can talk to the underlying storage system - obviously this requires that the block devices are network accessible via a protocol such as fibre channel or iSCSI, or that you have multiple hosts connected to the targets through the same SAS / SCSI bus. (GPFS also allows non-storage attached nodes to talk to the storage through storage-attached nodes). With GlusterFS / Lustre (not sure about Ceph) the data is stored on a host file system (such as XFS) and the GlusterFS / Lustre daemons manage and mediate access to it - but the data is only accessible through this daemon. The difference comes down to your access patterns and how you scale the systems. With GPFS you basically add more disks and storage systems (and network) to scale the IO, with GlusterFS / Lustre you add more servers (each with more disks). Usually, because servers with disks in them are cheaper than storage systems, the GlusterFS / Lustre style systems have lower capital costs. I suspect the running costs are higher due to inefficiencies in power/cooling for the servers, but I don't know this for sure. For Lustre, you need a minimum of two nodes, one for the objects (i.e. the data) and one for the metadata. I don't know if it's possible to collocate these functions on the same OS? If I remember correctly you can run GlusterFS and GPFS with just one node. Finally, for GlusterFS + XFS, there is a FAQ: http://community.gluster.org/q/whats-the-staus-of-xfs-support/ """ XFS is one of three supported filesystems for Gluster bricks, ext3 and ext4 are also supported. Using XFS does have size and performance implications that need to be considered. Gluster makes extensive use of extended attributes, XFS extended attribute performance in kernel versions < 2.6.39 is very poor. [1] This makes XFS a poor choice in environments where files are smaller than ~500MB """ Have fun! > Thanks, > > D. > > _______________________________________________ > Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf -- Jonathan Barber <jonathan.bar...@gmail.com> _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
