I have a home server using FreeBSD and ZFS that has worked well for the last 5 years, and on several occasions I have successfully replace faulty disks.
However, today a minor disaster has happened, and I'm hoping to find a solution.
I have a top-level pool that consists of 3 vdevs, each of which is a raidz1 pool, so up to 3 disks can fail -- assuming that they all belong to different vdevs -- and data integrity is intact.
Yesterday, I noticed quite a few errors being reported by 1 disk in 1 vdev. From past experience, this usually indicates that the disk is about to fail, so I do what I usually do:
- Offline the disk: zpool offline tank gpt/ta4
- Physically replace the disk
- Set up the new disk with gpart, and then zpool replace tank gpt/ta4
However, this time between steps 2 and 3 disaster struck: when I powered on the server after installing the new drive, I smelled something burning, and my HBA indicated that 4 of the drives weren't available! By a stroke of unbelievably bad luck, there must have been some voltage surge, because another drive in the same vdev (gpt/ta2) is now completely dead, and visual inspection reveals one of the MOSFETs on the PCB is blown.
So now gpt/ta2 is UNAVAIL and gpt/ta4 is OFFLINE, so obviously the vdev, which is raidz1, is also UNAVAIL.
My questions are: 1) Is there a way to bring gpt/ta4 back online? When I try to issue "zpool online tank gpt/ta4", it tells me that the pool is unavailable, so I can't do so. I can understand why this may be so, but I was thinking that gpt/ta4, although experiencing some read errors, was basically still a 'good' member of the raidz1 pool prior to taking it offline (zpool status reported there were no known data errors). Is there anyway to achieve this?
2) Failing that, is there a way to at least bring the remainder of my top-level pool (which consists of 3 raidz1 vdevs) online? The other 2 vdevs are perfectly fine.
Please help, I have a lot of precious data on it :-)
Thanks in advance.