Rsync is a fast and extraordinarily versatile file copying tool. It can copy locally, to/from another host over any remote shell, or to/from a remote rsync daemon

learn more… | top users | synonyms

4
votes
2answers
7k views

Speed up rsync by running multiple instances at once?

I have to transfer a big directory to another server. My problem is that i cannot use the full bandwidth that would be available, because one concurrent tcp stream does not get as fast. Basically ...
4
votes
2answers
6k views

ssh is ok, but connection refused when trying to rsync, why?

Editing the question, sorry for the confusion I need make cron job that will backup a directory that resides on a remote server, have the backup on the machine running the cron job. The remote server ...
4
votes
1answer
966 views

rsync copies unmodified files between different file systems

I try to synchronize files between two machines using rsync, but it seems to copy all the files regardless file modification time. Two consecutive call of the command produces the same results, so it ...
4
votes
2answers
112 views

executing a command again once it stops (rsync, FreeBSD)

I started doing backups via rsync to a remote server. This works all fine and good but 40 gig take their time and as I get a dynamic IP it gets interrupted every day. So every morning when I wake up I ...
4
votes
3answers
4k views

How to delete empty source directories when moving folders with rsync?

rsync can be coaxed into moving folders very similarly and superiorly to traditional mv with the following options: rsync -axvvES --remove-source-files source_directory /destination/ However one ...
4
votes
3answers
1k views

Forcing rsync to convert file names to lower case

We are using rsync to transfer some (millions) files from a Windows (NTFS/CYGWIN) server to a Linux (RHEL) server. We would like to force all file and directory names on the Linux box to be lower ...
4
votes
1answer
1k views

Using rsync to only delete extraneous files

What's the best way of comparing two directory structures and deleting extraneous files and directories in the target location? I have a small web photo gallery app that I'm developing. Users add ...
4
votes
2answers
8k views

How can I prevent vanishing error when using rsync with an nfs mount?

I'm in the process of moving files from an Ubuntu server to a Snow Leopard server. The Ubuntu server has an NFS share of around 6TB which I'd like to clone to the Snow Leopard server. I mounted the ...
4
votes
2answers
6k views

Rsync fails to backup some files, giving error 22

I'm trying to backup my whole server using a simple rsync on a NFS mounted volume on '/nas' Here's my rsync command rsync -sav -S --stats -H --numeric-ids --delete -D ...
4
votes
3answers
3k views

Chunking large rsync transfers?

We use rsync to update a mirror of our primary file server to an off-site colocated backup server. One of the issues we currently have is that our file server has > 1TB of mostly smaller files (in ...
4
votes
1answer
2k views

Using a shell script to check if rsync made any changes

I've got a bash script that rsync's two directories. Sometimes there's a change, odds are there's nothing. I want to run a command only if rsync actually made a change (add/update a file). ...
4
votes
2answers
6k views

Freenas — Choosing the right redundant storage regime (RAID, ZFS, Rsync?)

I've been running a freenas box for a few months as a proof of concept and all seems to be well now reliability wise. So it's time to up the storage capacity and build it into production mode. I was ...
4
votes
3answers
387 views

rsync many small files with long filenames takes a lot of bandwidth

I have a file storage server which stores files on disk using the file's sha256 hash as the filename, along with the file extension, and in three levels of directories, e.g. a PDF file with sha256 ...
4
votes
1answer
297 views

Preserving existing destination symlinks with rsync

I am using rsync to keep a local debian repository updated. Recently, the disk I am using to store it started running low on space, so I decided to use symlinks to allow me to move some of the ...
4
votes
1answer
1k views

Why does rsync forks itself? And why one such forked process is almost kinda idle (as seen in iotop)?

This is referring to the matter stated here and I am also experiencing the same. In one of my servers, I ran an rsync, to backup a huge directory (size greater than 300 Gb) to a different disk, ...
4
votes
3answers
2k views

Backup strategy for eCryptfs

We have a small network with a Ubuntu server in the middle and Ubuntu notebooks around. The notebooks all use eCryptfs to encrypt the whole home directory of users. Usually, there are two or more ...
4
votes
3answers
2k views

rsync and --remove-source-files

I use rsync v. 3.0.4 and when I need to move something I use it with the --remove-source-files. I prefer rsync than mv. Unfortunately, when I use --remove-source-files, the directories are left on ...
4
votes
1answer
586 views

how to backup svn remotely without a local copy

I have a subversion server that is running short on disk. I'd like to archive one of the larger repositories to free up some space, so I need to make a full backup of the repository files on a remote ...
4
votes
2answers
752 views

synchronizing storage servers

we have a storage server, with currently an amount of about 20TB of media files which we want to synchronize with a second storage server, for backup and fail-over. the facts are: we're storing ...
4
votes
1answer
1k views

rsync / rsnapshot

I am using rsnapshot and my config is as follows: config_version 1.2 snapshot_root /home/user/.snapshots/ cmd_cp /bin/cp cmd_rm /bin/rm cmd_rsync /usr/bin/rsync cmd_ssh /usr/bin/ssh ...
4
votes
2answers
2k views

Multi Thread Rsync Transfer

For some reason when running a single rsync command I am getting 1 MB/sec to 2 MB/sec even when I connecting 2 servers both connected to 1 Gbps ports. rsync -v --progress -e ssh ...
4
votes
2answers
4k views

RSync over SSH - permission denied even though the user is in the root group

I have a need to copy files between servers through the web. I'm using RSYNC over ssh to do so. The problem is, I need to be able to transfer files, no matter where the files is. I created a user ...
4
votes
2answers
2k views

rsync via multiple ssh connections with a key

I need to do a rsync from a target server with a server in the middle and I have the same public key on both the target and middle server. I want to use that key so I don't have to do password ...
4
votes
1answer
2k views

Linux server sync to an Amazon S3 bucket

I am looking for a stable solution to replace a classic server backup to another server using rsync. I have to sync a whole filesystem (more than 1Tb) to Amazon S3. Where am I? Solution 1: I mapped ...
4
votes
2answers
744 views

rsync not removing single quotes around array of strings --filter list

Trying to safely retrieve files from heavily infected Windows XP and 2003 machines that came to us from an acquisition. I have a similar need for automating a script that would run on PXE booted ...
4
votes
3answers
226 views

Backup Permissions for Entire File System

Here's the skinny: I have a Linode VPS that I want to backup to my local laptop via rsync. I have just generated ssh keys for the communication and left a blank passphrase for the private key so I can ...
4
votes
1answer
324 views

Rsnapshot hard-link ONLY + diffs

We have a 39 TB linux server (mdadm raid 6 15 3TB Drives) that has about 12TB of working data on it. Initially I wanted to create a snapshot system that only saved the data that was changed, much like ...
4
votes
5answers
11k views

rsync --files-from or --include-from

I tried the command: rsync -v --include-from=/path/to/list.lst /home/user /path/to/backup list.lst contains for example .gnupg/ .pki/ .gnome2/keyrings/ ...
4
votes
1answer
505 views

How can you prevent Permissions Problems when Restoring a Windows Machine with Rsync?

When restoring files (using rsync over cygwin) from Linux to Microsoft Windows XP (and probably to all version of windows), each file restored has permissions so strict, that only the "SYSTEM" account ...
4
votes
1answer
171 views

Cygwin rsyncd sends too many file names

I have rsyncd running as a Windows service, but when I try to access a module and sync from it, I get the files I want, as well as a lot that I do not — because they are not located in that module on ...
4
votes
1answer
193 views

rsync to adapt links that have absolute paths to the location at destination

Would it be possible with rsync to adapt the absolute, symbolic links to a new location at the destination? Test case Imagine I have a link pointing to a file like this: ...
4
votes
1answer
3k views

rsync error: unexplained error (code 130 ) at rsync.c(543) [sender=3.0.7]

rsync error: unexplained error (code 130 ) at rsync.c(543) [sender=3.0.7] I am having this error on a companys machien. unfortunately i don't have the exact version right now because i am already at ...
4
votes
2answers
941 views

syncing local folders

What is a best way to keep in sync two local folders ? I know I can use rsync -Ra but I need to keep in sync all the time. Is it possible to make rsync sit in background and constantly sync local ...
4
votes
1answer
505 views

Cygwin's RSYNC for large data transfer

I'm using rsync from Cygwin to do a large scale data transfer from an aging HP MSA 1000 to a new DAS attached to a different server. I have a daemon running on the remote server in read only mode and ...
4
votes
0answers
145 views

Server hacked - want to run an rsync backup in reverse. Will this work? [duplicate]

Possible Duplicate: My server's been hacked EMERGENCY My user account on a shared hosting server got hacked and overwrote every PHP with this at the bottom.... ...
4
votes
3answers
1k views

Have an unprivileged non-account user ssh into another box?

I know how to get a user to ssh into another box with a key: ssh -l targetuser -i path/to/key targethost But what about non-account users like apache? As this user doesn't have a home directory to ...
3
votes
5answers
6k views

SVN Backup using rsync command

I setup a cron for taking backup of my SVN repo( 8 GB) to another server. But some times I get errors and I feel that this is not the proper way to back an svn to a remote server. I used the command ...
3
votes
3answers
5k views

simple rsync in crontab without password

I would like to run the following: (Centos 5.4) rsync -avz /source_folder 192.168.1.1:/destination_folder right now I'm running it from CLI but eventually I would like to run it from crontab ...
3
votes
2answers
9k views

rsync “Operation not permitted”

I am trying to so a simple rsync to a USB hard disc, the command is rsync -az --modify-window=2 /home /mnt/exthd/tmp/ I am running this as root. but I get a lot of errors that all seem to relate ...
3
votes
5answers
1k views

Stop rsync from sending confirmation e-mails

Today I found out I have a 25GB /var/mail/root full of e-mails from rsync. Apparently it's sending myself (root@server) an e-mail with the file list every time I run an rsync, which happens to be once ...
3
votes
4answers
5k views

Can I rsync from Putty on windows to my Cent OS server?

I'm a bit new to the command line and UNIX but am finding it hugely useful. Is it possible for me to rsync from putty on my windows 7 machine to my virtual server? If not what are my other command ...
3
votes
1answer
346 views

How do I check if a backup by rsync is successful?

I use rsync -a to do a full backup on a file system. Its output to stderr is empty. Shall I do some additional checking to verify if the backup is successful? For example, how can I do a checksum to ...
3
votes
4answers
4k views

rsync for backups: without --remove option and rotations…?

Is reasonably safe to use rsync (without --delete option) to make my daily backups and keep only 1 copy on the backup server (instead of having multiple copies with rotations) ? If I don't enable the ...
3
votes
6answers
708 views

Need to check a directory tree for changes every 10 seconds and run a script if it is changed

I have a directory that has many folders with code inside it, and I need to run a script to rsync changes over to a couple other servers. I have that script written and working fine, but I cannot ...
3
votes
2answers
2k views

rsync with script supplied password

I need to have a script periodically rsync files from a remote host via ssh. The account which the script runs under does not have home directory on the remote host, so I'm not able to store my ...
3
votes
3answers
631 views

What is the best way to copy 9TB of data to multiple external drives, sequentially?

The data is currently stored in a 9TB RAID array, with many users having many files and directories. We'd like to back it up to five 2TB external hard drives, but since we only have one free USB port, ...
3
votes
1answer
8k views

Force rsync to copy symlink, even if referent does not exist

I am getting this message with rsync: symlink has no referent: "/a/path" Well, rsync is right: in the source filesystem, the (symbolic) link is pointing to a non-existent location. That is ...
3
votes
3answers
2k views

Rsync - maintain 'root' permission on another machine?

I'm just trying to copy a directory from one home server to another, both running Ubuntu (9.04 and 9.10), using rsync. The trouble I'm having is that some of the files in the directory (and ...
3
votes
3answers
1k views

How to keep multiple servers in sync file wise?

I'm currently managing a cluster of PHP-FPM servers, all of which tend to get out of sync with each other. The application that I'm using on top of the app servers (Magento) allows for admins to ...
3
votes
4answers
2k views

Optimizing RAID 5 for BackupPC use (small random reads)

I am running a BackupPC server with a hardware RAID 5 for the main storage of the backups. Since the machine was created on a tiny budget, the controller is a 3Ware 9500S-4LP for the PCI port and the ...