Server Fault is a question and answer site for system and network administrators. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I am running FreeBSD 9.1 with root on a ZFS mirrored pool.

At first the pool (labelled zroot) has a total of 1TB spanning two 1TB drives.

Recently we have run into disk space issues in which the pool is 99% allocted with about 15G free. About 800G is occupied in /home filesystem alone.

To remedy this, we have added 2x 1TB drives.

We would like to keep these drives mirrored. The question now remains, how should we add the additional 2x 1TB drives to the pool?

  1. Should we add them to the existing zroot pool and allocate it. This seems like a bad idea since the original pool was designed to run as the root filesystem.

  2. Should we create a new pool with a mirrored set of drives, and then create a new /home2 and transition high-usage users to that partition?

To us, the extent of ZFS features isn't clear. We are leaning to option (2) since it doesn't add unnecessary dependencies on the root mirror. However, we would appreciate any insight and ideas on what course of action to take.


share|improve this question
up vote 4 down vote accepted

Should we add them to the existing zroot pool and allocate it. This seems like a bad idea since the original pool was designed to run as the root filesystem.

First, zpools and zfs file systems are separate concepts (you can't actually separate them, but should when thinking about their setup). So pools are not "designed to run as the root filesystem", there is a filesystem on the zpool that is configured to run as the root filesystem.

Being that this zpool holds the root filesystem, it has to either be non-redundant single vdev, or has to be a single mirror. So you can't just add more drives.

Should we create a new pool with a mirrored set of drives, and then create a new /home2 and transition high-usage users to that partition?

It's be much easier to create zfs filesystems for each user, and mount it to their home drive location. This eliminates the janky/non-standard /home2 idea, and allows you to set quotas, reservations, and more on each user's home folder.

When I setup a ZFS server, I either use two/three smaller disks for the boot zpool (I've got 100-320GB disks laying around, so that's what I commonly use), or slice a chunk off the main storage drives (of about the same size, using GPT partitions). The rest of the drive get's one large partition, and that goes into a storage zpool.

Here's my home server setup for example:

  pool: tank0
 state: ONLINE
  scan: scrub repaired 0 in 1h39m with 0 errors on Sat Jun  8 07:59:40 2013

        NAME              STATE     READ WRITE CKSUM
        tank              ONLINE       0     0     0
          mirror-0        ONLINE       0     0     0
            gpt/tank0-d0  ONLINE       0     0     0
            gpt/tank0-d1  ONLINE       0     0     0

errors: No known data errors

  pool: tank1
 state: ONLINE
  scan: scrub repaired 0 in 3h11m with 0 errors on Mon Jun  1 08:57:39 2013

        NAME              STATE     READ WRITE CKSUM
        tank1             ONLINE       0     0     0
          raidz1-0        ONLINE       0     0     0
            gpt/tank1-d0  ONLINE       0     0     0
            gpt/tank1-d1  ONLINE       0     0     0
            gpt/tank1-d2  ONLINE       0     0     0
            gpt/tank1-d3  ONLINE       0     0     0
share|improve this answer
Always avoid jankiness. Creating the individual filesystems makes much more sense. – ewwhite Jun 11 '13 at 13:45
Said FreeBSD version can boot off raidz zpool. – kworr Jun 11 '13 at 15:04
Thanks for pointing out the conflation of pool and filesystem. sometimes I feel like ZFS just has too many moving parts. I will migrate each user into a separate filesystem -- this will hopefully make storage management a bit more flexible I'm not sure why I didn't consider it in the first place. Thanks. – n n Jun 12 '13 at 15:21
It's not entirely obvious, so many people miss it. There's a ZFS Admin Guide that covers a lot of best practices and methodologies. If you haven't read it already I highly recommend at least reading the first 3 chapters, they're full of good information. – Chris S Jun 12 '13 at 15:50

First of all let me guess your current configuration. Did you use zpool add to add new drives to the pool? If so your zpool should look like:


In this case to mirror your new drives you should do this:

zpool attach <pool-name> <disk3> <disk5>
zpool attach <pool-name> <disk4> <disk6>

This will give you a pool consisting of three mirrors two drives per each.

In my opinion it would be much better to use zraid2 or zraid3 for this setup, you'll get a faster pool and better reliability. In your current setup you can lose two drives to kill a pool. With zraid2 you can also lose two drives but you'll get 1TB extra space. With zraid3 you can safely lose three drives.

Another option you can consider is enabling compression on your pool. Do not enable compression for the root file system!

share|improve this answer
There's no such thing as "zraid2/3", they're "raidz-2" and "raidz-3" respectively. They are absolutely slower than mirrors, and provide little additional redundancy. Also, the configuration sample you provided isn't possible as zpools holding root file systems have to be either a single disk or a mirror. – Chris S Jun 11 '13 at 13:33
I agree they are referred as raidzN in manual. However they are absolutely faster then mirrors or at least my setup performs so. There's also no restriction on using raidz for root fs, it's rather restriction for bootloader only and my setup perfectly boots off raidz2 pool. – kworr Jun 11 '13 at 15:02
Apparently fBSD 9.x comes with a new bootloader that support raidz, my apologies. – Chris S Jun 11 '13 at 16:41
raidz generally beats mirrors in raw throughput tests, especially at large block sizes. Mirrors beat the everloving piss out of raidz in anything else, especially small block and/or IOPS-constrained tasks. Why no compression for the root filesystem? Nexenta has been compressing the live root filesystem for some time now, as a default. – Nex7 Jul 25 '13 at 18:55

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.