0

I have a volume group data containing one logical volume data, composed of currently 10 drives (total 13TB), of which one physical volume is failed (/dev/sdi, 1TB) and one is its replacement drive (/dev/sdl, 2TB). The data volume is mirrored (usable size 5TB).

$ sudo pvs
  PV         VG              Fmt  Attr PSize    PFree   
  /dev/sdc   data            lvm2 a--    <2.73t   <2.73t
  /dev/sdd   data            lvm2 a--   931.48g  931.48g
  /dev/sde   data            lvm2 a--    <2.73t       0 
  /dev/sdf   data            lvm2 a--  <465.73g <465.73g
  /dev/sdg   data            lvm2 a--    <1.82t   <1.82t
  /dev/sdh   data            lvm2 a--   931.48g       0 
  /dev/sdi   data            lvm2 a-m   931.48g       0 
  /dev/sdj   data            lvm2 a--   931.48g       0 
  /dev/sdk   data            lvm2 a--  <465.73g <465.73g
  /dev/sdl   data            lvm2 a--    <1.82t  931.50g

$ sudo vgs
  VG              #PV #LV #SN Attr   VSize   VFree 
  data             10   1   0 wz-pn-  13.64t <7.28t

$ sudo lvs
  LV   VG              Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  data data            rwi-aor---  <4.55t                                    100.00          
 

I shouldn't have lost any data and the mounted volume behaves correctly. I inserted the drive to replace the failing drive and attempted to pvmove /dev/sdi it and it failed at 16%. Now pvmove and vgreduce complain about the unfinished pvmove and mention a volume pvmove0 which I guess is a temporary volume for the move operation, and we can see this pvmove0 volume in pvdisplay under the 2 concerned drives sdi and sdl.

$ sudo pvmove /dev/sdi
  WARNING: Missing device /dev/sdi reappeared, updating metadata for VG data to version 18.
  WARNING: Device /dev/sdi still marked missing because of allocated data on it, remove volumes and consider vgreduce --removemissing.
  Cannot change VG data while PVs are missing.
  Consider vgreduce --removemissing.
  Cannot process volume group data
  WARNING: Missing device /dev/sdi reappeared, updating metadata for VG data to version 18.
  WARNING: Device /dev/sdi still marked missing because of allocated data on it, remove volumes and consider vgreduce --removemissing.
  Cannot change VG data while PVs are missing.
  Consider vgreduce --removemissing.
  Cannot process volume group data
  Failed to find physical volume "/dev/sdi".
  Run `pvmove --help' for more information.

$ sudo vgreduce --removemissing data
  WARNING: Partial LV pvmove0 needs to be repaired or removed. 
  There are still partial LVs in VG data.
  To remove them unconditionally use: vgreduce --removemissing --force.
  WARNING: Proceeding to remove empty missing PVs.

$ sudo lvdisplay --maps
...
  --- Logical volume ---
  LV Path                /dev/data/data
  LV Name                data
  VG Name                data
  LV UUID                RJLfTh-DPiI-mSLf-UJPi-lYd7-CDds-neGkjx
  LV Write Access        read/write
  LV Creation host, time HomeServer, 2019-11-20 14:01:42 +0100
  LV Status              available
  # open                 1
  LV Size                <4.55 TiB
  Current LE             1192305
  Mirrored volumes       2
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:7
 
  --- Segments ---
  Logical extents 0 to 1192304:
    Type        raid1
    Monitoring      monitored
    Raid Data LV 0
      Logical volume    data_rimage_0
      Logical extents   0 to 1192304
    Raid Data LV 1
      Logical volume    data_rimage_1
      Logical extents   0 to 1192304
    Raid Metadata LV 0  data_rmeta_0
    Raid Metadata LV 1  data_rmeta_1
 
$ sudo pvdisplay --maps
...
  --- Physical volume ---
  PV Name               /dev/sdi
  VG Name               data
  PV Size               931.48 GiB / not usable 1.71 MiB
  Allocatable           yes (but full)
  PE Size               4.00 MiB
  Total PE              238459
  Free PE               0
  Allocated PE          238459
  PV UUID               4wpS3t-Gg1n-iFhi-Yi0y-IpeQ-0Vas-kLPT4X
 
  --- Physical Segments ---
  Physical extent 0 to 238458:
    Logical volume  /dev/data/pvmove0
    Logical extents 0 to 238458
...
  --- Physical volume ---
  PV Name               /dev/sdl
  VG Name               data
  PV Size               <1.82 TiB / not usable <1.09 MiB
  Allocatable           yes 
  PE Size               4.00 MiB
  Total PE              476924
  Free PE               238465
  Allocated PE          238459
  PV UUID               Ou36N2-I7PS-vCsZ-d89m-Duk0-qSuw-n5BXJP
 
  --- Physical Segments ---
  Physical extent 0 to 238458:
    Logical volume  /dev/data/pvmove0
    Logical extents 0 to 238458
  Physical extent 238459 to 476923:
    FREE

What can I try to fix it ?

Host info:

$ sudo lvm version
  LVM version:     2.02.186(2)-RHEL7 (2019-08-27)
  Library version: 1.02.164-RHEL7 (2019-08-27)
  Driver version:  4.34.0
  Configuration:   ./configure --build=x86_64-redhat-linux-gnu --host=x86_64-redhat-linux-gnu --program-prefix= --disable-dependency-tracking --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib64 --libexecdir=/usr/libexec --localstatedir=/var --sharedstatedir=/var/lib --mandir=/usr/share/man --infodir=/usr/share/info --with-default-dm-run-dir=/run --with-default-run-dir=/run/lvm --with-default-pid-dir=/run --with-default-locking-dir=/run/lock/lvm --with-usrlibdir=/usr/lib64 --enable-lvm1_fallback --enable-fsadm --with-pool=internal --enable-write_install --with-user= --with-group= --with-device-uid=0 --with-device-gid=6 --with-device-mode=0660 --enable-pkgconfig --enable-applib --enable-cmdlib --enable-dmeventd --enable-blkid_wiping --enable-python2-bindings --with-cluster=internal --with-clvmd=corosync --enable-cmirrord --with-udevdir=/usr/lib/udev/rules.d --enable-udev_sync --with-thin=internal --enable-lvmetad --with-cache=internal --enable-lvmpolld --enable-lvmlockd-dlm --enable-lvmlockd-sanlock --enable-dmfilemapd

$ uname -a
Linux server 4.4.205-1.el7.elrepo.x86_64 #1 SMP Fri Nov 29 10:10:01 EST 2019 x86_64 x86_64 x86_64 GNU/Linux

$ cat /etc/redhat-release 
CentOS Linux release 7.8.2003 (Core)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.