How to set up disk cloning with dd, netcat and ssh tunnel?

Question

I would like to copy stuff in bulk (reimage disk using dd) with netcat from host A to B via ssh encrypted channel on Linux.

What commands should I type on both ends? Could you please explain what each parameter means? I was googling for a while, but could not find a concise walkthrough.

Thank you!

Answer 1

Copying from source to target where target has sshd running:

• dd if=/dev/sda | gzip | ssh root@target 'gzip -d | dd of=/dev/sda'

Copying from source to target via sshd_host when target is not running sshd.

• Target: nc -l -p 62222 | dd of=/dev/sda bs=$((16 * 1024 * 1024))
• Source: ssh -L 62222:target:62222 sshd_host &

• Source: dd if=/dev/sda | nc -w 3 localhost 62222

  dd - if= is the source, of= is the destination, bs= is the block size. Different block sizes may improve performance. 16 is usuually a fairly reasonable starting point. You can also use count= to indicate how many blocks to copy.

  nc - -p indicates the port to use for services. -l is used to start a service. -w sets up the time to wait for data in the pipline before quiting.

  ssh - -L sets up the tunnel on the remote host. The format of the argument is, local_port:target_host:target_port. Your local program (nc) connects to the local_port, this connection is tunneled and connected to target_port on the target_host.

The options defined are just the ones used for this. Look at the man pages for more details.

A few notes:

1. If you are doing this over anything but a LAN, I'd suggest compressing the datastream with gzip or compress. Bzip2 would work too but it takes a bit more CPU time. The first one has an example of that usage.
2. Its better if the source partition is not mounted or is mounted read-only. If not you will need to fsck the destination image.
3. Unless one of the machines has netcat but not ssh, netcat isn't really needed here. That case would look like:

source machine dd -> nc -> ssh -> ssh tunnel -> sshd server -> nc on target -> dd

1. dd works best if the source and targets are the same size. If not the target must be the bigger of the 2.
2. If you are using ext2/3 or xfs, dump (or xfsdump) and restore may be a better option. It wont handle the boot sector but it works when the target and source are different sizes.

Answer 2

Or, you could use clonezilla and "mount" your remote storage through sshfs.

Answer 3

If you wanna use netcat without ssh. I presume that is the fastest way and not the secure one, you can copy and restore the whole disk like this:
On computer A with IP 192.168.0.1

cat /dev/hdb | nc -p 9000

On computer B

nc -l 192.168.0.1 9000 > /dev/hdb

Remember that according to man nc the -l option is:

  -l  Used to specify that nc should listen for an incoming connection rather than initiate a connection to a remote host.  It is an error to use this option in conjunction with the -p, -s, or -z options.

Answer 4

The basic copy with netcat is described here.

If you need to get SSH involved in to this, you could use port forwarding over that,

-R [bind_address:]port:host:hostport

But, on the whole, you could just do the SSH transfer in the first place (without netcat).

Answer 5

netcat is not needed.

on src machine run:

dd if=/dev/sdX bs=1M | ssh root@dstMachine " dd of=/dev/sdY bs=1M"

i assume none of partitions on sdX and sdY are mounted. you can boot both boxes with knoppix or other similar live distro.

dd - takes data from if [ if not provided - takes it from stdin ], sends data to of [ if not provided - data is sent to stdout ]. bs - block size... will speed things out.

ssh - executes command provided in quotes on remote box, all data pumped to stdin of ssh will be tunneled to remote machine and porovided as stdin to command executed there.

Answer 6

So long as the filesystems are both unmounted, dd works well.

(from server1) dd if=/dev/sda bs=32k | ssh <server2> dd of=/dev/sda bs=32k

You'll need hostkey authentication setup ahead of time or else the password prompt will cause the copy to fail.

Doing this on a mounted volume will produce poor results.

Answer 7

Host A is the one to image, host B is the one the image will be stored on:

root@A# dd if=/dev/sda | ssh root@B "dd of=/some/file"

Restoring to disk would just swap those two.

Answer 8

Looks like you're using a sledgehammer to crack a nut here - or perhaps a better analogy is trying to cut your lawn with scissors :)

I would strongly suggest you look at some of the tools out there for doing a job like this unless you've got great reasons to do it in-house.

Trinity Rescue Kit is a free liveCD which supports imaging drives over multicast, and might do what you want (or indeed anyone else thinking on the same lines), without going to full-bore imaging systems.