# How to copy a large LVM volume (14TB) from one server to another?

I have to copy a very large LVM volume from server A to server B. Below is the filesystem of server A and server B

Server A

[root@AVDVD-Filer ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg_avdvdfiler-lv_root
16T   14T  1.5T  91% /
tmpfs                 3.0G     0  3.0G   0% /dev/shm
/dev/cciss/c0d0p1     194M   23M  162M  13% /boot
/dev/mapper/vg_avdvdfiler-test
2.3T  201M  2.1T   1% /test
/dev/sr0              3.3G  3.3G     0 100% /mnt


server B

[root@localhost ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup-LogVol00
20G  2.5G   16G  14% /
tmpfs                 3.0G     0  3.0G   0% /dev/shm
/dev/cciss/c0d0p1     194M   23M  162M  13% /boot
/dev/mapper/VolGroup00-LogVol00
16T  133M   15T   1% /xiangao/lv1
/dev/mapper/VolGroup00-LogVol01
4.7T  190M  4.5T   1% /xiangao/lv2


I want to copy the LVM volume /dev/mapper/vg_avdvdfiler-lv_root on server A to LVM volume /dev/mapper/VolGroup00-LogVol00 on server B. Server A and server B are in the same IP segment. In the LVM volume on server A, there is all average 500M avi wmv mp4 etc.

I tried mounting /dev/mapper/vg_avdvdfiler-lv_root on server A to server B through NFS, then use cp to copy. It is clear I failed.

Because the LVM volume is too big, I do not have good idea why. I hope a good solution here.

-

A tar pipe is probably best for this.

-
How does a tar pipe handle open/changing files, in comparison to let's say rsync? 14TB move is obviously not going to be an atomic operation ;) –  Marcin Aug 22 '12 at 17:28
rsync enumerates all the files beforehand and doesn't like change. tar will (try to) read each of the files in turn regardless of state. –  Ignacio Vazquez-Abrams Aug 22 '12 at 17:56

i think i would use rsync over ssh in this situation.

if the transfer dies half-way, using tar over ssh, you won't be able to resume the transfer, you will have to start from scratch.

-
thanks , I'll try the rsync over ssh . I hope it cann't dies half-way. –  bruce May 13 '10 at 16:25
even if it dies, you can restart it and continue from there on –  golemwashere Aug 22 '12 at 17:46
+1 for ability to resume through use of rsync. You could also use rsync's options for compression which would help a bit. If you must maintain hard links across this transfer, you need to be careful of RAM usage when using -H option. Rsync also tries very hard to ensure the target file is same as source file if you use the checksum option - see notemagnet.blogspot.co.uk/2009/10/… - which is important given many TB of data. (Ideally you would have ZFS on both ends for end to end checksumming.) –  RichVel Oct 31 '12 at 16:16

This was the best solution for me:

dd if=/dev/myvolumegroup/mylogicalvolume bs=4096 | pv | ssh targetmachine dd of=/dev/myvolumegroup/mylogicalvolume bs=4096


You have to setup the lvm logical volume on the target machine first, with the same or more space.

Note, some people have suggested this with nc (netcat), however it won't work because netcat will close when it receives an EOF, of which there are many within a partition. Ssh doesn't have this problem.

I used bs=4096 instead of bs=512 because it was much faster. YMMV.

The reason pv is in there is that it shows you progress, which is pretty nice when sending large amounts of data so you get an idea of transfer rates and how long you expect to wait before it's finished.

-
From the netcat docs on the sourceforge project page, nc seems to do the opposite when an EOF is received: " This continues indefinitely, until the network side of the connection shuts down. Note that this behavior is different from most other applications which shut everything down and exit after an end-of-file on the standard input." –  gravyface Sep 30 '12 at 9:58
Never heard of pv before, nice tool :) –  flob Oct 23 '12 at 8:52

If you want to copy the CONTENTS of the volume group I would suggest using tar or dump/restore piped over SSH (create an appropriate volgroup on Server B & restore the data) - see the man pages for those tools for more info.

If you want to copy the volume group itself (structure as well as contents), ask yourself why & insist on a good reason. If you come up with compelling reasons look into drbd or something similar

-
Of course，I only want copy CONTENTS of the volume . Although I do not know the Dump/restore utilities, but I'll try . Thank you very much ! –  bruce May 13 '10 at 16:11

Your english isn't too bad, I think everyone will be able to understand.

To clarify one thing, do you want the contents of "vg_avdvdfiler-lv_root" to be transferred, or do you want the LVM itself (which might include things like snapshots)?

-
I only want the contents of "vg_avdvdfiler-lv_root" to be transferred –  bruce May 13 '10 at 16:16

If you were feeling adventurous, you could set up DRBD between the two machines. This would bring the two in sync in the background.