Hot answers tagged

183 votes

Rsync triggered Linux OOM killer on a single 50 GB file

So let us read the oom-killer output and see what can be learned from there. When analyzing OOM killer logs, it is important to look at what triggered it. The first line of your log gives us some ...
  • 40.5k
94 votes

Can I make rsync output only the summary?

Thanks to a tip by Wayne Davison, I use the --stats option for backup: rsync -a --stats src/ dest/ Nice little summary at the end, e.g. Number of files: 6765 Number of files transferred: 0 Total file ...
  • 1,042
55 votes

rsync files to a kubernetes pod

To rsync to a Pod I use the following helper. pod=$1;shift;kubectl exec -i $pod -- "$@" I put this in a file called "" and then run the rsync like so. rsync -av --...
54 votes

Rsync to AWS S3 bucket

To communicate to s3 you need to have 2 things IAM user credentials who has read-write access to s3 bucket. A client like aws-cli for bash, boto library for Python etc. once you have both, you can ...
48 votes

rsync: getcwd(): No such file or directory (2)

I had the exact same issue, and my solution was something I had never thought of. In my case rsync worked fine for all the time until it stopped during some tests after changes to one of my scripts. ...
45 votes

Using rsync to only delete extraneous files

You need --existing too: rsync -r --delete --existing --ignore-existing /home/gallery/images /home/gallery/thumbs From the manpage: --existing, --ignore-non-existing This tells rsync to ...
  • 1,562
44 votes

Faster rsync of huge directory which was not changed

Some unrelated points: 80K is a lot of files. 80,000 files in one directory? No operating system or app handles that situation very well by default. You just happen to notice this problem with rsync....
  • 7,767
38 votes

Using rsync from msysgit for binary files

As of now (9/7/2017) it is extremely easy to manually add rsync support to the Git for Windows environment without even having to resort to running the Git for Windows SDK. I stumbled across this ...
35 votes

How do I set destination permissions with rsync (chown chmod)

rsync needs to be told that you want to set the permissions and owner/group information. It would be logical to assume that having --chmod or --chown would tell that but they don't. For permissions ...
31 votes

Rsync to AWS S3 bucket

You can try rclone. Setup (docs): rclone config Sync (docs): rclone sync /my/local/folder s3service:bucket-name
  • 1,215
28 votes

Is it possible to use rsync over sftp (without an ssh shell)?

An alternative to using rsync is to instead use lftp (which can connect to sftp) and use the mirror command. For example one can do lftp ~> open -u user,password s ~> mirror ...
  • 549
28 votes

Perform rsync while following sym links

You need both -L and -K if you have symlinks on both sides, e.g. you already had your 1st rsync done and want to update the backup using rsync. -L, --copy-links transform symlink into ...
  • 381
22 votes

Perform rsync while following sym links

Just ran into this problem. And if you want rsync to treat symlinked directories as directories, you want the K option rsync -K /files/ user@server:/files/
  • 341
22 votes

Run totally silent rsync?

quiet option should do that: -q, --quiet suppress non-error messages
22 votes

rsync - exclude all directories except a few

A simple filter should do the trick. To build on the prior answer with a proper example -- Explicitly include the parent(s), plus all (**) sub folders and files. Then exclude everything else. Here's ...
20 votes

rsync --compress-level: which compression levels can be used?

It's values between 0-9. Where 1 is fastest and 9 is the most compressed. Other than that there's correlation between rsync and zlib where rsync tells the zlib library to "use the default ...
  • 424
19 votes

How to keep the full path with rsync?

Another common related use case it to keep a selected part of the directory tree with /./, e.g. if you wanted: /home/udi/files/pictures to go to: backup/files/pictures you can write: rsync --...
19 votes

Restricting a ssh key to only allow rsync/file transfer?

rrsync is designed to be used as a forced command for a particular key, so it should be exactly what you want. A forced command is set up using the command option for a key in an authorized keys file ...
16 votes

Preserve ownership with rsync, without root

As stated in the other answers, to directly preserve ownership information you need root access to the destination machine. However, you had at least two workarounds to avoid root access while ...
  • 45.5k
16 votes

rsync error: error in rsync protocol data stream (code 12) at io.c(600) [sender=3.0.6]

rsync is not installed on 999.999.99.9
15 votes

Copying a large directory tree locally? cp or rsync?

This thread was very useful and because there were so many options to achieve the result, I decided to benchmark few of them. I believe my results can be helpful to others have a sense of what worked ...
  • 251
15 votes

Rsync daemon: is it really useful?

rsyncd (rsyncd with daemon) uses by default port 873. This will use the native rsync protocol, but does not encrypt anything. It is meant only for local or trusted networks. If you want security, ...
  • 151
15 votes

Can rsync display current average speed?

In case you're not running v3.1.0, rsync-3.0.9 takes the --progress flag. The man page on my CentOS7 box reports This option tells rsync to print information showing the progress of the transfer. ...
  • 345
15 votes

rsync : write failed - No space left on device (28)

I had the same issue, the destination directory had enough space but I'd get "No space left on device". Turns out rsync copies the file to somewhere else first, then moves it to the destination ...
15 votes

Does rsync change file content (dealing with hardlinks)?

With standard options, rsync will copy the new file assigning it a semi-random name, then it will rename the new file with the original name. In this process, no writes are directed at the original ...
  • 45.5k
14 votes

How to delete empty source directories when moving folders with rsync?

I ended up in a similar situation. I wanted to avoid rm -rf, knowing that rsync failed to copy some files, and I didn't want to loose them. To simply delete the empty directories, I found this one ...
  • 241
14 votes

How to tell rsync do not check permissions

If you supply the -a flag -p is automatically included. If you specify --no-p all of the other options from -a remain intact.
  • 241
14 votes

rsync output file sizes in human-readable format

Reading the --human-readable section of the man page reveals -h, --human-readable Output numbers in a more human-readable format. This makes big numbers output using larger units, with a K, M,...
  • 115k
13 votes

How do "correctly" sync millions of files with rsync and inotify?

Your case sounds exactly what lsyncd was made for. It watches the directory with inotify and spawns an rsync with only the changed file(s). It aggregates changes for a few seconds. So when you make 5 ...
  • 17.4k
12 votes

rsync - report only uploaded files

rsync with a single -v|--verbose actually prints only transferred files (as well as a header/footer). If you are getting the complete list every time then it probably means that the default rsync src/...

Only top scored, non community-wiki answers of a minimum length are eligible