How to upgrade a ZFS RAID-Z array to larger disks on OpenSolaris?

Question

I'm planning on installing OpenSolaris on my home server (right now it has Linux) and I would like to know how to prepare for upgrading the server to have larger hard drives in the future. Now the server has 4x 400GB SATA drives and I would install OpenSolaris with ZFS RAID-Z on them. Some time later this or next year I'm going to upgrade the drives to maybe 4x 1TB SATA drives.

How can I do the upgrading and increasing of the partition size with ZFS? Is it a simple and painless operation (e.g. replace one drive at a time and ZFS will automatically resize itself to use the full space), or are some special steps required? Can it be done without shutting down the system?

Answer 1

1. Offline drive #1.
2. Replace the old drive with the newer, high-capacity drive.
3. zpool replace the drive.
4. Wait for the replacement to be rebuilt.
5. Goto step 1 and repeat for each successive drive in the array.

You may need to export and then import the zpool once this procedure is complete; the pool will then show the increased size. No downtime is required, but if a drive does go south while you're upgrading your array, you will lose everything, as with four total disks you presumably have a 3+1 setup. (The odds aren't that high, but Murphy never sleeps.)

The official source: "Replacing Devices in a Storage Pool" in the ZFS Administration Guide. The ZFS Best Practices Guide is also useful.

Answer 2

I would like to add that replacing a non defective drive makes the raid less fail proof. You can actually fill the replacing drive first with data before taking out the smaller drive. So the order will become.

• Attach bigger drive (can even be through usb)
• zpool replace smallerdrive biggerdrive
• wait
• when it is done take out smaller drive, or whatever you need to do.
• repeat
• and at the end you might need to run zpool set autoexpand=on pool

Now murphy is less of a problem