http://blogs.sun.com/constantin/entry/opensolaris_home_server_zfs_and
 A couple of weeks ago, OpenSolaris 2008.05, project Indiana,
saw its first official release. I've been looking forward to this
moment so I can upgrade my home server and work laptop and start
benefiting from the many cool features. If you're running a server at
home, why not use the best server OS on the planet for it?
This
is the first in a small series of articles about using OpenSolaris for
home server use. I did a similar series some time ago and got a lot of
good and encouraging feedback, so this is an update, or a remake, or
home server 2.0, if you will.
I'm not much of a PC builder, but Simon
has posted his experience with selecting hardware for his home server.
I'm sure you'll find good tips there. In my case, I'm still using my
trusty old Sun Java W1100z workstation, running in my basement. And for
storing data, I like to use USB disks.
USB disk advantages
This
is the moment where people start giving me that "Yeah, right" or "Are
you serious?" looks. But USB disk storage has some cool advantages:
- It's cheap. About 90 Euros for half a TB of disk from a
major brand. Can't complain about that.
- It's hot-pluggable.
What happens if your server breaks and you want to access your data?
With USB it's as easy as unplug from broken server, plug into laptop
and you're back in business. And there's no need to shut down or open
your server if you just want to add a new disk or change disk
configuration.
- It scales. I have 7 disks running in my
basement. All I needed to do to make them work with my server was to
buy a cheap 15 EUR 4-port USB card to expand my existing 5 USB ports. I
still have 3 PCI slots left, so I could add 12 disks more at full USB
2.0 speed if I wanted.
- It's fast enough. I measure about
10MB/s in write performance with a typical USB disk. That's about as
fast as you can get over a 100 MBit/s LAN network which most people use
at home. As long as the network remains the bottleneck, USB disk
performance is not the problem.
ZFS and USB: A Great Team
But
this is not enough. The beauty of USB disk storage lies in its
combination with ZFS. When adding some ZFS magic to the above, you also
get:
- Reliability. USB disks can be mirrored or
used in a RAID-Z/Z2 configuration. Each disk may be unreliable (because
they're cheap) individually, but thanks to ZFS' data integrity and
self-healing properties, the data will be safe and FMA will
issue a warning early enough so disks can be replaced before any real
harm can happen.
- Flexibility.
Thanks to pooled storage, there's no need to wonder what disks to use
for what and how. Just build up a single pool with the disks you have,
then assign filesystems to individual users, jobs, applications, etc.
on an as-needed basis.
- Performance. Suppose you upgrade
your home network to Gigabit Ethernet. No need to worry: The more disks
you add to the pool, the better your performance will be. Even if the
disks are cheap.
Together, USB disks and ZFS make a great team. Not enterprise class,
but certainly an interesting option for a home server.
ZFS & USB Tips & Tricks
So
here's a list of tips, tricks and hints you may want to consider when
daring to use USB disks with OpenSolaris as a home server:
- Mirroring vs. RAID-Z/Z2: RAID-Z (or its more reliable
cousin RAID-Z2)
is tempting: You get more space for less money. In fact, my earlier
versions of zpools at home were a combination of RAID-Z'ed leftover
slices with the goal to squeeze as much space as possible at some
reliability level out of my mixed disk collection.
But say you have
a 3+1 RAID-Z and want to add some more space. Would you buy 4 disks at
once? Isn't that a bit big, granularity-wise?
That's why I decided
to keep it simple and just mirror. USB disks are cheap enough, no need
to be even more cheap. My current zpool has a pair of 1 TB USB disks
and a pair of 512 GB USB disks and works fine.
Another advantage of
this aproach is that you can organically modernize your pool: Wait
until one of your disks starts showing some flakyness (FMA and ZFS will
warn you as soon as the first broken data block has been repaired).
Then replace the disk with a bigger one, then its mirror with the same,
bigger size. That will give you more space without the complexity of
too many disks and keep them young enough to not be a serious threat to
your data. Use the replaced disks for scratch space or less important
tasks.
- Instant replacement disk: A few weeks ago, one of
my mirrored disks showed its first write error. It was a pair of 320GB
disks, so I ordered a 512GB replacement (with the plan to order the
second one later). But now, my mirror may be vulnerable: What if the
second disk starts breaking before the replacement has arrived?
That's
why having a few old but functional disks around can be very valuable:
In my case, took a 200GB and a 160GB disk and combined them into their
own zpool:
zpool create temppool c11t0d0 c12t0d0
Then, I created a new ZVOL sitting on the new pool:
zfs create -sV 320g temppool/tempvol
Here's out temporary replacement disk! I then attached it to my
vulnerable mirror:
zfs attach santiago c10t0d0 /dev/zvol/dsk/temppool/tempvol
And
voilá, my precious production pool stated resilvering the new virtual
disk. After the new disk arrived and has been resilvered, the temporary
disk can be detached, destroyed and its space put to some other good
use.
Storage virtualization has never been so easy!
- Don't forget to scrub:
Especially with cheap USB disks, regular scrubbing is important.
Scrubbing will check each and every block of your data on disk and make
sure it's still valid. If not, it will repair it (since we're mirroring
or using RAID-Z/Z2) and tell you what disk had a broken block so you
can decide whether it needs to be replaced or not just yet.
How
often you want to or should scrub depends on how much you trust your
hardware and how much your data is being read out anyway (any data that
is read out is automatically checked, so that particular portion of the
data is already "scrubbed" if you will). I find scrubbing once every
two weeks a useful cycle, othery may prefer once a month or once a week.
But
scrubbing is a process that needs to be initiated by the administrator.
It doesn't happen by itself, so it is important that you think of
issuing the "zpool scrub" command regularly, or better, set up a
cronjob for it to happen automatically.
As an example, the following line:
23 01 1,15 * * for i in `zpool list -H -o name`; do zpool scrub $i; done
in your crontab will start a scrub for each of your zpools twice a
month on the 1st and the 15th at 01:23 AM.
- Snapshot often:
Snapshots are cheap, but they can save the world if you accientally
deleted that important file. Same rule as with scrubbing: Do it. Often
enough. Automatically. Tim Foster did a great job of
implementing an automatic
ZFS snapshot service, so why don't you just install it now and set
up a few snapshot schemes for your favourite ZFS filesystems?
The
home directories on my home server are snapshotted once a month (and
all snapshots are kept), once a week (keeping 52 snapshots) and once a
day (keeping 31 snapshots). This gives me a time-machine with daily,
weekly and monthly granularities depending on how far back in time I
want to travel through my snapshots.
So, USB disks aren't
bad. In fact, thanks to ZFS, USB disks can be very useful building
blocks for your own little cost-effective but reliable and
integrity-checked data center.
Let me know what experiences you
made while using USB storage at home, or with ZFS and what tips and
tricks you have found to work well for you. Just enter a comment below
or send me email!
"OpenSolaris
Home Server: ZFS and USB Disks" has been brought to you by Constantin's Blooog.
This entry was created on 2008-05-27 13:40:54.0 PST and is associated
with the following tags:
disks howto opensolaris server solaris usb zfs
You're welcome to use this Permalink
, add a comment below or send your feedback to constantin at sun dot
com.
|
