One of the three disks went off in my LVM raid1 logical volume. (It was a bad experiment actually). The volume is a pure LVM
raid1 one (no, not an LVM
mirror), without any mdadm / fakeraid.
The problem was realized only at reboot, the device was completely missing. As I still have two healthy disks, I'd like to use the volume as a two-way raid1 for a while, and to replace the bad disk later. However, I cannot remove the missing device from the volume.
Here is the basic info about the topology. The volume group is
vgQ3, while the logical volume is
#lvm pvs | grep vgQ3 Couldn't find device with uuid vG3BbG-ap9H-iYGg-qfE2-R13Z-rLJI-AJcAv0. /dev/sdb2 vgQ3 lvm2 a-- 79.31g 17.91g /dev/sdc2 vgQ3 lvm2 a-- 79.31g 17.91g unknown device vgQ3 lvm2 a-m 79.31g 17.91g # lvm lvs -a -o +devices | grep vgQ3 Couldn't find device with uuid vG3BbG-ap9H-iYGg-qfE2-R13Z-rLJI-AJcAv0. lvRAID1 vgQ3 rwi---r-p 61.39g lvRAID1_rimage_0(0),lvRAID1_rimage_1(0),lvRAID1_rimage_2(0) [lvRAID1_rimage_0] vgQ3 Iwi---r-- 61.39g /dev/sdc2(1) [lvRAID1_rimage_1] vgQ3 Iwi---r-- 61.39g /dev/sdb2(1) [lvRAID1_rimage_2] vgQ3 Iwi---r-p 61.39g unknown device(1) [lvRAID1_rmeta_0] vgQ3 ewi---r-- 4.00m /dev/sdc2(0) [lvRAID1_rmeta_1] vgQ3 ewi---r-- 4.00m /dev/sdb2(0) [lvRAID1_rmeta_2] vgQ3 ewi---r-p 4.00m unknown device(0)
First I tried to reduce the volume group, but without success:
# vgreduce --removemissing vgQ3 Couldn't find device with uuid vG3BbG-ap9H-iYGg-qfE2-R13Z-rLJI-AJcAv0. WARNING: Partial LV lvRAID1 needs to be repaired or removed. WARNING: Partial LV lvRAID1_rmeta_2 needs to be repaired or removed. WARNING: Partial LV lvRAID1_rimage_2 needs to be repaired or removed. There are still partial LVs in VG vgQ3. To remove them unconditionally use: vgreduce --removemissing --force. # vgreduce --removemissing --force vgQ3 Couldn't find device with uuid vG3BbG-ap9H-iYGg-qfE2-R13Z-rLJI-AJcAv0. Persistent log is not supported on segment-by-segment mirroring
And so on... Then I tried to reduce the three-way raid1 device to a two-way one (Trying to reduce it to a one-way linear volume resulted in the same error message):
# lvconvert -m1 vgQ3/lvRAID1 Couldn't find device with uuid vG3BbG-ap9H-iYGg-qfE2-R13Z-rLJI-AJcAv0. Cannot change VG vgQ3 while PVs are missing. Consider vgreduce --removemissing.
Well, I've run out of the ideas I could try, except that destroying the volume and rebuilding it, which considerably would be more painful than simply reducing it to a two-way raid1.
(To make the story complete, by executing:
# vgchange -a y -P vgQ3
I could create and mount the raid1 volume (/dev/vgQ3/..) in partial mode, so I could save the content. The next reboot makes the device disappear however, so the story continues again and again).
Is there any solution to such problems?