Rsync is a fast and extraordinarily versatile file copying tool. It can copy locally, to/from another host over any remote shell, or to/from a remote rsync daemon

learn more… | top users | synonyms

3
votes
3answers
6k views

simple rsync in crontab without password

I would like to run the following: (Centos 5.4) rsync -avz /source_folder 192.168.1.1:/destination_folder right now I'm running it from CLI but eventually I would like to run it from crontab ...
132
votes
16answers
199k views

How to use rsync over FTP

Any unix: I have the following cmd line which works fine. rsync -avr -e ssh /home/dir user@example.com:/home/ But I need to set it up now to rsync to a remote server that only has an FTP server on ...
35
votes
5answers
41k views

Is it possible to use rsync over sftp (without an ssh shell)?

Rsync over ssh, works great every time. However, trying to rsync to a host which allows only sftp logins, but not ssh logins, provides the following error: rsync -av /source ssh ...
8
votes
3answers
6k views

Why don't people just use rsync to back up vmware guests?

If I am running a modern vmware ESXi system, I can drop in a statically linked rsync binary and rsync files to any destination over SSH. I'm trying to understand why most (all ?) backup of vmware ...
4
votes
3answers
11k views

Complex includes/excludes with rsync

I'm trying to work out the rsync filter syntax to perform complex include/excludes, and trying to achieve the following: Include / Exclude /home Include /home/user1/* Include /home/user2/subdir/* ...
3
votes
2answers
1k views

Git-style incremental backup with rsync

I'm trying to setup a backup script on Ubuntu. Every day I want to copy my local source directory to a backup directory on a remote server uniquely named with the date. (e.g., backup-jan1/, ...
2
votes
2answers
2k views

Possibility of WAN Optimization for SSH traffic

While I do understand that SSH by itself is a very low bandwidth utilizing protocol, it is sometimes choking up bandwidth during peak hours in our office environment. I want to know if at all it is ...
62
votes
3answers
4k views

Rsync triggered Linux OOM killer on a single 50 GB file

I have a single 50 GB file on server_A, and I'm copying it to server_B. I run server_A$ rsync --partial --progress --inplace --append-verify 50GB_file root@server_B:50GB_file Server_B has ...
62
votes
18answers
68k views

Why hasn't rsync caught on in the Windows world? [closed]

It seems that rsync is the de-facto standard for efficient file backup and sync in Unix/Linux. Does anyone have any thoughts on why it wouldn't have caught on in the Windows world? Why hasn't it ...
18
votes
13answers
11k views

Complete restore of linux system

I am familiar with using rsync to back up various files from my system but what is the best way to completely restore a machine. What I have tried in the past is: Do a basic format/reinstall from ...
28
votes
3answers
45k views

Why is my rsync so slow?

My Laptop and my workstation are both connected to a Gigabit Switch. Both are running Linux. But when I copy files with rsync, it performs badly. I get about 22 MB/s. Shouldn't I theoretically get ...
23
votes
9answers
25k views

Best way to compare (diff) a full directory structure?

What's the best way to compare directory structures? I have a backup utility which uses rsync. I want to tell the exact differences (in terms of file sizes and last-changed dates) between the source ...
13
votes
8answers
15k views

Copy large file from one Linux server to another

I'm attempting to copy a 75 gigabyte tgz (mysql lvm snapshot) from a Linux server in our LA data center to another Linux server in our NY data center over a 10MB link. I am getting about 20-30Kb/s ...
30
votes
6answers
30k views

protocol version mismatch — is your shell clean?

When following the instructions to do rsync backups given here: http://troy.jdmz.net/rsync/index.html I get the error "protocol version mismatch -- is your shell clean?" I read somewhere that I ...
15
votes
3answers
12k views

compare 2 directories and copy differences in a 3rd directory

Running ubuntu 12.04, I want to compare 2 directories, say folder1/ and folder2/ and copy any files that are different to folder3/. There are also nested files, so matching subdirectories should be ...
10
votes
3answers
10k views

Rsync seems incompatible with .bashrc (causes “is your shell clean?”)

It turns out rsync can't work with a remote server which has a .bashrc file? At local client i got when run rsync: protocol version mismatch -- is your shell clean? (see the rsync man page for an ...
5
votes
3answers
8k views

Escaping spaces in a remote path when using rsync over a remote SSH connection

When using SSH to connect rsync to a remote server, how do you escape spaces and such in the remote path? A simple backslash escapes the space for the local bash prompt, but on the remote machine the ...
16
votes
2answers
13k views

How to repeatedly call rsync until files are sucessfully transferred

I'm trying to syncronize files from a remote server that is not reliable, meaning the connection tends to fail "randomly" with rsync: connection unexpectedly closed Rsync is called with ...
10
votes
6answers
8k views

what is the fastest and most reliable way of transferring a lot of files?

I'm trying to transfer about 100k files totaling 90gb. Right now I'm using rsync daemon but its slow 3.4mb/s and I need to do this a number of times. I'm wondering what options do I have that would ...
10
votes
2answers
331 views

Server migration: most efficient way

I have been tasked with migrating one of our sites between servers (two different hosts). Both environments are linux. The site streams video, so the server is currently filled with media files ...
8
votes
5answers
12k views

How does RSYNC does incremental Backups

How does RSYNC knows which files are chnages and which are not. Does it logs its data anywhere in the file. Because I want to do incrrmental backups but first it will transfer all files. So my main ...
5
votes
7answers
13k views

Speeding up rsync over smb

I'm backing up a Linux box over SMB to a NAS. I mount the NAS locally and then I rsync a lot of data (100GB or so). I believe it's taking an awfully long time to do it: more than 12 hours. I would ...
3
votes
1answer
166 views

Limited SSH access for log retrieval

I want to have limited access to a Linux server to be able to grab a certain set of logs. Ideally, I would like to be able to run rsync locally to grab the logs from the server and for security ...
3
votes
2answers
2k views

rsync with script supplied password

I need to have a script periodically rsync files from a remote host via ssh. The account which the script runs under does not have home directory on the remote host, so I'm not able to store my ...
2
votes
2answers
1k views

How do I reserve bandwidth for rsync?

Currently nginx uses all my bandwidth. How do I leave some for rsync?
4
votes
2answers
2k views

rsync via multiple ssh connections with a key

I need to do a rsync from a target server with a server in the middle and I have the same public key on both the target and middle server. I want to use that key so I don't have to do password ...
3
votes
1answer
972 views

Rsync: dealing with absolute symlinks

Having read through the rsync manpage, I can't see any way for rsync to automagically transform absolute symlinks between machines, i.e Server A file -> /home/username/path/file Server B file ...
2
votes
7answers
3k views

Backup - rsync or tar

We're looking to backup about 100gb+ of data containing small files (10kb+) each. The backup needs to be done as fast as possible to another harddrive weekly. Which is the better (especially speed ...
1
vote
1answer
929 views

Improve RSYNC scan speed / Alternate approach?

I'm using rsync to transfer application log files from one server to another. It works fine with the exception that scanning the directories takes forever. I have around 160,000 log files and a few ...
1
vote
2answers
7k views

rsync error unexplained error (code 255) at io.c

I was using a script to perform rsync in sudo crontab. The script does a 2-way rsync (from serverA to serverB and reverse). After i reboot both the server machines, the rsync is not working in sudo ...
1
vote
3answers
2k views

How to make an ISO copy of Linux-filesystem and user files of VPS Debian based?

I have a Debian-Based VPS on some hosting. I want to migrate from it and i need to make a full copy of all Linux-filesystem (and installed packages) + all home directory with website files. And then ...
0
votes
6answers
2k views

realtime sync of files between instances

In my scenario, i have 2 ec2 instances and need to sync the files between them in realtime. Can I do this with rsync? Can anyone please suggest how to schedule a rsync?
0
votes
2answers
89 views

Need help w/ rsync

I need some help getting my rync command syntax working. I need to clone the /home/user/public_html folder on a remote server to the server I am logged into via SSH, copying the folder into the same ...
130
votes
13answers
145k views

Copying a large directory tree locally? cp or rsync?

I have to copy a large directory tree, about 1.8 TB. It's all local. Out of habit I'd use rsync, however I wonder if there's much point, and if I should rather use cp. I'm worried about permissions ...
94
votes
16answers
84k views

Showing total progress in rsync: is it possible?

I have searched for this option already, but have only found solutions that involve custom patching. The fact that it does not show in --help and no more info can be found probably indicates the ...
79
votes
3answers
49k views

What is archive mode in rsync?

I know you can use -a or --archive to activate archive mode when using rsync. Unfortunately, I have no idea what archive mode is supposed to do, and the man page is not at all explicit about what this ...
17
votes
9answers
24k views

Sync LVM snapshots to backup server

I have a number of Xen virtual machines running on a number of Linux servers. These VMs store their disk images in Linux LVM volumes with device names along the lines of /dev/xenVG/SERVER001OS and so ...
20
votes
5answers
12k views

Rsync : copying over timestamps only

Currently I have two directories A/ and B/ which are identical in every respect, with the exception of the timestamps. Therefore if I run the command : rsync --dry-run -crvv A/ B/ then all files ...
8
votes
5answers
12k views

ZFS Sync over unreliable, slow WAN. ZFS replication, or rsync?

I've been tasked with making an off-site backup work over the WAN. Both storage boxes are FreeBSD based NAS boxes running ZFS. Once or twice a week, 15-60 gigs of photography data gets dumped to the ...
13
votes
4answers
17k views

rsync is just hanging, building file list

[root@centos /]# rsync -av --exclude thumbs /storage root@xx.27.1.xx:/storage root@xx.27.1.xx's password: building file list ... been sitting for about an hour... it's 135GB of images and folders ...
12
votes
3answers
12k views

How to perform a secure rsync between servers across an unsecured network

Basically what I'm asking is, has anyone come across a means by which to wrap rsync inside ssh. With OpenSSH v4.9+ sftp has some nice options that allow you to chroot the incoming connection and such ...
12
votes
4answers
7k views

Converting UTF-8 NFD filenames to UTF-8 NFC, in either rsync or afpd

I have a home file server running FreeNAS 8. A few days ago I used rsync to upload my entire iTunes library from Mac so that I could load my library over the network instead of off a slow USB drive. ...
8
votes
10answers
29k views

Using Rsync to backup to an external drive

I'm buying an external hard drive to back up the computers in my house (finally!!). I'm hoping to use rsync. I've seen an example that does (or seems to do) exactly what I want. Something like ...
16
votes
5answers
10k views

How can I use rsync with a FAT file system?

I would like to write a simple backup script that saves some data to a FAT drive. Should I reformat the drive and use a better file system or is it possible to use rsync with FAT? If so, what problems ...
14
votes
3answers
15k views

Maximizing rsync performance and throughput - directly-connected gigabit servers

I have two Dell R515 servers running CentOS 6.5, with one of the broadcom NICs in each directly attached to the other. I use the direct link to push backups from the main server in the pair to the ...
8
votes
3answers
22k views

Rsync command issues, owner and group permissions doesn´t change

I´m trying to set owner and group via rsync and it doesn't seem to be working. This is the command: sudo rsync -rlptDvz --owner=cmsseren --group=cmsseren /home/serena/public_html/ -e ssh ...
9
votes
4answers
2k views

Linux to Linux, 10 TB transfer?

I've looked at all the previous similar questions, but the answers seemed to be all over the place and no one was moving a lot of data (100 GB is different from 10 TB). I've got about ...
6
votes
3answers
7k views

Trying to setup chroot'd rsync

I'm trying to set up a backup server. I want to chroot each user (client) to its home directory, and only allow it to use sftp and rsync. I quickly discovered that I was not the only one trying to do ...
3
votes
1answer
1k views

Rsnapshot to multiple destinations (or maintain hard links structure)

I'm doing a backup of multiple external servers on a local machine using rsnapshot and it's working beautifully. Now, I'd like to clone the destination snapshots on an external disk that will be ...
2
votes
1answer
6k views

rsync and --fake-super - how to preserve attributes?

I have been trying to sort out my backups, so that when files are restored the owners/groups and any other metadata are correctly restored. For my testing I have been backing up from a Ubuntu 9.10 ...