Convert full hard drive to LVM without external storage

Question

Recently, my 2TB hard drive ran out of space, so I purchased another one to supplement it. My old drive is mounted in a specific folder (does not have system files or anything). Technically, I could just make a new folder and use that as the mountpoint for the new drive, but it would be much neater if I could use LVM to combine the two drives so that there is only one mountpoint.

Is there a way to convert my existing 2TB drive to LVM live? I do not have any external storage to use as a transfer medium nor are there any other backup mediums that will suffice, so I need a way to convert it without actually deleting anything on it (and the drive is full so I cannot create a new partition and use that as backup either).

Answer 1

Nope, you can't convert an existing full drive to LVM without temporarily moving your data somewhere else. You need to create one or more fresh partitions on the drive, and then run pvcreate on the new filesystem to create an LVM physical volume. Then you create and activate a volume group on top of that, and so on.

However, you say that you bought a new drive so I'm assuming it's at least 2TB, right? That provides you a migration path if you can get both drives running in your system at the same time.

First, connect the big new drive as your second (I'll assume sata) drive. It will come up on the system as /dev/sdb. Use fdisk to create two partitions on it. One should be a small boot partition, a gig is probably plenty. Devote the rest to a new lvm partition. Use the lvm tools (pvcreate, vgcreate, etc.) to set up LVM on that second drive. Leave the small boot partition empty.

Shut down the machine and disconnect your old 2TB drive. Move the new drive to the primary sata interface, or use the bios to make it primary. Boot the machine using your install dvd. Install a new linux system on just the small boot partition you created.

Once you have that working, reconnect your original drive, now on the secondary sata channel. Bring the system up and configure your new LVM-based filesystem as /newdrive (on /dev/sda now, remember). Mount the old drive somewhere like /olddrive.

Copy all the data off the old hard drive onto your new drive by suning something like rsync to copy the contents of /olddrive to /newdrive. Now you have transitioned all your data to the new LVM filesystem.

Now comes the scary part: wipe your original drive. Think very carefully about this because if you wipe the wrong device you lose all your original data. Perhaps you could borrow another drive form someone else to serve as a temporary backup?

Once the original drive is blank, set it up as an LVM device with pvcreate. Then you can use vgextend and lvresize to grow your LVM volume group onto that new device. That can be done while the system is running and will not affect your data on the new drive. Once that operation is done, you have a 4TB volume group. You can use resizefs on the resulting filesystem to make it fill the whole space.

Note that there are many ways to screw this oepration up, but it is feasible. I strongly encourage you to make extra backups somehow. Also note that one giant volume group running on top of two separate drives is not very robust - if one drive fails you could lose a lot of data. A better idea is often to run lvm on top of a raid zero array of two mirrored drives. That way you only get 2TB of space, but one drive dying doesn't cause everything to completely fail. Here's how I performed an LVM migration a while back, for reference.

Another option is to install a third drive in the machine and use it just for booting. That makes things a little simpler since you don't have to set up seperate regular and lvm partitions on your boot device. Actually modern Linux distros may have fixed this but I know a few years back that boot loaders like grub couldn't work with lvm so yo always had to boot off a non-lvm device.

Answer 2

Looks like I figured it out on my own: I created a volume group with just the new hard drive, so I can copy the data to the new drive, then wipe the old drive and just add it to the volume group.

Answer 3

The question is looking for in-place conversion without moving lots of data around. If permanently copying 4 MiB (the size of 1 LVM extent) to the new disk is acceptable it can be done without shrinking the filesystem (a requirement of blocks or manual lvmify from https://superuser.com/questions/493481/fastest-way-to-convert-an-ext4-formatted-disk-to-lvm-with-ext4-on-it):

1. Make a backup of the first 16 MiB of the old partition as it may come in handy in case of mistakes. Also make a backup of the last 8 MiB if the size of the partition is not a multiple of 4 MiB, e.g. with tail --bytes. Note down the exact size of the partition. Consider doing a practice run using a device snapshot of the data partition, e.g. use https://gist.github.com/jowagner/b36024636140ddf453c12eaf6e590b5d

2. Create a PV and VG on the new disk and create an LV with size 4 MiB (1 extent).

3. Restore or copy the first 4 MiB of the old partition to the new LV. Verify that cmp /dev/mapper/vgname-lvname /dev/sdX1 (adjust names to your setting) reports "EOF [...] after byte 4194304" (4 MiB is 4194304 bytes).

4. Create a PV with --dataalignment 4M on the old partition. This will overwrite a small fraction of data in the first 1 MiB but we have 2 backups from steps 1 and 3. (In case pvcreate detects a signature and offers to wipe it, we don't know whether it also wipes some data beyond 4M so you may prefer to first wipe the first 4M manually so pvcreate does not detect your payload format.)

4b. Add the new PV to the volume group created in step 2 with vgextend.

5. Run lvextend to grow the LV to add all extents of the PV on the old partition to the LV, e.g. query the free space on the PV in MiB and specify this amount in -l+1234MiB. The PV must be specified in the command to allocate from the target PV. Verify with cmp against the backup in step 1. It should report "EOF [...] after byte 16777216" as we backup up 16 MiB and 16 MiB is 16777216 bytes.

6. If the old partition's size was not a multiple of 4 MiB there is still a small amount of data missing in our new LV (called "data LV" below). Otherwise continue with step 7.

   a) Create an LV with just 1 extent (this should automatically go on the new disk's PV as the PV on the old disk should no longer have any free extents at this stage) called "tiny LV" below.

   b) Copy the missing data to the tiny LV. This is the size of the old partition modulo 4 MiB and since it is a multiple of 512 bytes and not a multiple of 4 MiB it must be between 512 and 4,193,792 bytes. The data can be sliced from the backup with tail --bytes.

   c) Remove the tiny LV.

   d) Grow the data LV to absorb the extent we just prepared.

   e) Verify success with tail --bytes 4194304 and cmp against the data slice prepared in 6.2. It should report EOF with the length of the slice.

7. For a first test, you may want to create a copy-on-write snapshot and run a full filesystem check and/or data scrubbing without modifying any more data on the old partition. Delete the snapshot to maximise free space for the next step.

8. Finally, the LV can be extended and the filesystem grown to use the available space on the new disk.

9. If this was a dry-run on a device snapshot, run lvremove, vgremove and pvremove to unregister the LVM devices, remove the snapshot and repeat all steps this time for real. (It's easy to forget this when seeing that everything works as desired except for eventually running out of snapshot space or, in case of a non-permanent snapshot, loosing all new data when powering down.)

Filesystem Journal Considerations

With the above setup, the first 4 MiB of the filesystem will be on the new disk while the second 4 MiB are on the old disk. Many filesystems keep a journal near the start of the filesystem. You may prefer to have more than the first 4 MiB on the new disk, i.e. creating a larger LV in step 2. The backup in step 1 must then be larger as well. The LV can even be a striped LV if step 4 is moved before step 2. (To extent a striped LV with a linear segment, use --stripes 1.) In both cases an additional padding LV will have to be created on the old disk's PV so that step 5 skips data that is already in the LV. This padding LV can be removed after step 7.

Warnings

• I haven't tested the exact steps above yet, only a similar approach in a slightly different setting.

• All the above relies on knowing that LVM only overwrites metadata at the start of each PV, that extents are by default allocated sequentially and that extents are 4 MiB in size. Future versions of LVM may change any of these.

Answer 4

Yes it can be done, it is old question but I needed to reply as you can convert an existing hard drive to LVM without moving your data and nobody has said it can be done. You need to create one new LVM partition on new disk and then run pvcreate with -Zn on the old filesystem to create an LVM physical volume but rememeber to use -Zn when creating vg or lv other wise you will lose all your data and if you have brought new 2 TB hard disk your total disk will be 2+2 TB.