Reducing an lvmgroup - cannot resize to xxxx extents as later ones are allocated

Question

I've been going through the process of reducing the size of a Centos 6.10 image by reducing an lvmgroup. I'm at the point where I've been able to run this command:

pvresize -tv --setphysicalvolumesize  1600G /dev/md125p2

And it's giving me the following error message:

 /dev/md125p2: cannot resize to 409599 extents as later ones are allocated.

I'm trying to understand how to construct the pvmove command to get rid of the error, but I don't quite get the concept.

Here is the output of this command pvs -v --segments /dev/md125p

  PV           VG            Fmt  Attr PSize PFree   Start  SSize  LV       Start Type   PE Ranges                 
  /dev/md125p2 vg_centos6svr lvm2 a--u 1.86t 320.11g      0 384000 lv_mysql     0 linear /dev/md125p2:0-383999     
  /dev/md125p2 vg_centos6svr lvm2 a--u 1.86t 320.11g 384000  81947              0 free                             
  /dev/md125p2 vg_centos6svr lvm2 a--u 1.86t 320.11g 465947  12800 lv_root      0 linear /dev/md125p2:465947-478746
  /dev/md125p2 vg_centos6svr lvm2 a--u 1.86t 320.11g 478747   1998 lv_swap      0 linear /dev/md125p2:478747-480744
  /dev/md125p2 vg_centos6svr lvm2 a--u 1.86t 320.11g 480745   7500 lv_home      0 linear /dev/md125p2:480745-488244

I know the move command has to look something like this:

pvmove --alloc anywhere /dev/md125p2 vg_centos6svr:yyyy-end

I'm just not clear how to calculate the yyyy-end part. I figure it means move that 81947 (blocks?) that start at 384000, but what is "yyyy" and "end"?

Answer 1

So, what I can see is everything is on /dev/md125p2 and we want to reduce the size of this PV.

Your output from pvs -v --segments /dev/md125p show you have a very large free space (SSize==81947) after your lv_mysql and before your lv_root. This is big enough to house lv_root and all of the LVs that come after it in the physical layout.

So, the first set of numbers I'd suggest is:

***# pvmove --alloc anywhere /dev/md125p2:480745-488244 /dev/md125p2:384000+7499

that is, move the whole of lv_home to the free space starting directly after lv_mysql.

Then re-run pvs -v --segments /dev/md125p to see the revised numbers, and move the next block into whatever free space is left.

This will effectively defrag your PV.

Now, pvmove may have way more options than what I'm using (maybe specifying the PV instead of the extents?) but what I got comfortable with is just being explicit with the blocks, so all of my sources and targets are in the form of /dev/<dev>:<Start>-<End> or /dev/<dev>:<Start>+<SSize-1>.

I suspect your problem was mainly missing the extents off of your "source" specification.

I've seen a few examples out their using -end - I don't know if that's something that's supposed to work, but it would beg the question "end of what?". So I avoid it and use explicit block numbers. You just need to be mindful that the + syntax is a plus, and needs to be one less than the SSize.

Extra

OK, on the source side it seems like you can do -name lv_home /dev/md125p2 as a short-hand for /dev/md125p2:480745-488244 - but note also that this will get more complex if lh_home was fragmented into multiple extents (would it mean "all fragments?") - I think I still prefer the numbers approach. Note that your example in the question didn't include the LV name, so I'm not sure what pvmove would have understood from what you gave it...

Answer 2

Reducing a PV requires moving extents to free space before the desired size. In your case extents below 409599.

First, test a backup restore to be sure data can be recovered if necessary.

lvremove volumes you do not need.

pvmove allows specifying extents by LV name, and the destination by offsets with the + notation. When providing the destination, that must be a PV, not a VG.

Move one LV to the first extent in free space. The offset is the size of the segment being moved.

pvmove --alloc anywhere  -n lv_swap /dev/md125p2 /dev/md125p2:384000+1998

Get the new segment layout with pvs -v --segments again. Repeat the pvmove with a different LV and the new free space start extent.

Personally, I prefer data VGs separate from operating system VGs. But this is your storage, lay it out as you wish.