Have you seen the Drobo box? it’s a SAN that allows you to create giant volumes and hot swap out hard drives at will with failure tolerance… bad news, is that it costs close to £1000 even without the drives, i’ll explain how to make a better one… for free! =).
ZFS (Zettabyte Filing System) is Sun’s newest file-system offering, its supported on FreeBSD / Solaris natively and Mac OS X / Linux / Windows via third-party utilities. I’m gonna keep this guide, simple, short and sweet, so i’ll bullet list the main features that wow people about ZFS =)
- It can store up to 340 quadrillion zettabytes of data (no other production filing system can do this)
- It checksum’s your data on the fly so you can check for integrity by “scrubbing” it (identifying broken drives before they completely die)
- It supports every raid configuration you can think of natively and doesn’t suffer from the raid5 data-hole.
- You can create snapshots of your data that do not waste hdd capacity.
- Volumes or “Pools” can be expanded at any time, so you can start with a 2tb raid, and increase it to a 10tb raid with no data loss.
- You can mix/match capacities, brands, rpm’s of drives.
- Its reliable* (on officially supported incarnations anyway)
- Its a memory whore (don’t try it unless you have 2gb ram on your system)
- Its supported in the latest version of FreeNAS (0.7)
- Allows hotplugging of drives when one fails (so you don’t lose data/time)
- Hotspares are supported
- Can be easily transferred / transported to any other ZFS supported system without extensive configuration or any data loss.
- Its free free free free (under CDDL).
Think of a hardware raid5 or a geom_concat/raid and then think about those again, but without any of the issues / flaws they have… thats what ZFS is! =)
So lets get started, I’ll run through creating and bringing a ZFS raid online first, and then some maintenance commands afterwards. I suggest trying this on a box with at least 1gb ram (2gb recommended) and an operating system with a 64bit kernel that ZFS is supported on nicely (FreeBSD / Solaris / FreeNAS).
In our configuration we have 3x 1tb drives which are called da0, da1, da2, and we are going to create a raidz (raid5 without data-hole) so we have 1 drive redundancy in the event of failure (meaning 1 hard drive can die and your data is safe), this will give us 2tb of available space rather than 3tb obviously.
There are two main commands used in zfs, (yes just two) these are, zpool (used to manage / create “pools”) and zfs (used for practically everything).
As root, run:
~# zpool create DATA raidz da0 da1 da2
This will create a ZFS data “pool” called DATA using a raidz configuration. Please note however, this also creates the virtual device needed for ZFS first (as pools are basically bags full of virtual devices & virtual devices are basically collections of drives). So in this command we have created both our virtual device encompassing da0,1,2 and also our pool “DATA”.
Now we can bring our ZFS raid online… yes really its that simple
~# zpool online DATA
However! this 2 step version will only work on recent incarnations of ZFS, on older ones you may have to run
~# zfs create DATA
Now if we run “zpool status” we should see the following:
~# zpool status
pool: DATA
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
DATA ONLINE 0 0 0
da0 ONLINE 0 0 0
da1 ONLINE 0 0 0
da2 ONLINE 0 0 0
errors: No known data errors
Similarly if we run “zpool list”:
~# zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
DATA 2.97T 220K 1.99T 0% ONLINE -
Showing that our 3tb raid is online but with 2tb available as the capacity of one drive is used for parity information…
And that is that! a working zfs system, type df -h and you will see it has been auto mounted probably somewhere like /{poolname},
Maintenance commands / tips & tricks
Ok lets say da1 has failed, you’re data is still ok, and intact, but its not safe anymore as if another drive fails, you will have lost everything, so we need to replace da1 quickly. This is easy
~# zpool offline DATA da1
Now put in the new drive and:
~# zpool replace DATA da1 new-device
new-device being the device name of the new drive… (this may be the same). Now ZFS will start to repopulate the new drive with data, which you can monitor using the command “iostat”. Once it has finished, your raid will have its integrity restored =). its that simple!
Scenario 2: You just purchased another 5x 1tb drives and want to add them to your existing ZFS pool “DATA” without losing any data… plug them in, and note down their device names (we’ll use wd0 wd1 wd2 wd3 wd4).
~# zpool add DATA raidz wd0 wd1 wd2 wd3 wd4
This will create another virtual device for the new drives and add them to the DATA pool in a raidz configuration, you’ll see your volume increase auto-magically without having to run any additional commands / growfs or anything!… note however, as you are effectively creating another set of redundancy, you will again lose the capacity of one of these drives, so you’ll end up with 4tb rather than 5tb. This ensures data integrity and protection upon hard drive failure.
Scenario 3: There was a thunder-storm, your computer crashed after a powercut, one of your drives now starts clicking and your’e worried if the data is ok… Remember i mensioned ZFS did live checksumming? here it is in action…
~# zpool scrub DATA
This will start the “scrubbing” process and daemonise it. As this can take some time (hours) just walk away, get some dinner and come back later… when you’re back run:
~# zpool status DATA
And look out for the line “errors:” this will say hopefully “No errors detected, last scrub was on some date” =)
So there you have it, oh and don’t forget, if you have FreeNAS you can do all of this from the web administration panel also! so no messing around required. Have a test and let me know what you think! Your comments are always welcome =)
