Tag Info

New answers tagged

1

The lastest source distribution of rsync contains this wrapperscript called rsync-no-vanished: #!/bin/bash IGNOREEXIT=24 IGNOREOUT='^(file has vanished: |rsync warning: some files vanished before they could be transferred)' set -o pipefail rsync "${@}" 2>&1 | (egrep -v "$IGNOREOUT" || true) ret=$? if [[ $ret == $IGNOREEXIT ]] then ret=0 ...


2

By default rsync will read the entire file on both source and destination, to verify that they are identical. This does not consume network bandwidth, as it will only be comparing a hash value. But it does spend time reading from the disk. In one usage scenario, I found this to be terribly inefficient because the source files were only being appended to. I ...


0

why do you say it takes longer each time? how is that possible? maybe it's the program generating the files that is touching them? try with --checksum: skip based on checksum, not mod-time & size, see if that changes anything (i wouldn't keep this option because it reads every file from the disk every time, too expensive, i'm only suggesting it to find ...


1

From your symptom, sync attempts were most likely unfinished due to slow link and the additional sync killing afterwards. In that case --inplace option is actually the worst option since it is not designed for network-bound situations. Quoting from the --inplace option: WARNING: The file’s data will be in an inconsistent state during the transfer (and ...


0

consider using --delete-after and not --delete, this will ensure that your receiving end will delete files after transfer, not before.


21

Rsync does a one way sync, however it's up to you to decide which way the sync goes. Rsync command syntax is the following: rsync [OPTION...] SRC... [DEST] Note that you specify sync from source to destination. Source and destination can be any local or remote path. For example if you want to copy files from your server to your laptop you do: rsync ...


1

The directory won't be deleted if permissions forbid it, or if the directory is not empty. Often there are files that can't be deleted because they are immutable or otherwise protected. What does ls -a /usr show on the destination? You might want to try the --force-delete flag, but first run it with -n (dry run mode) to see what it would delete.


2

Adding --inplace will help with big files greatly. It makes picking up after a failed or killed rsync faster because it doesn't use temporary files.


1

You can not rsync repository per se as directory at FS, because at physical layer it doesn't correlate with logical content. You must to get content of repository (better - without SVN-related metadata) as real tree on server-side before and rsync this tree Using (any) SVN-client (CLI svn may already exist or you have to install it), you'll svn export from ...


1

Rsyncing a single file is not a backup solution, what do you do when something happened to the vm and files were deleted, but you only noticed this after your rsync has run again? You will have overwritten the good 'backup' of your files with the bad image now. If you want backup you need to keep the old versions somewhere, or the diff's. Rsync will only ...


2

I added the --ignore-existing command and it looks like it won't change anything and only download new files. rsync -vzr --ignore-existing -e Edit: When there are new files it still takes longer each cycle.


1

It depends... If changing the permissions result in rsync being unable to read and/or write on your webserver the job should fail. Normally a change of permissions is handled intelligently and rsync will only change the permissions if the file hasn't changed, without transferring the file itself. The option --no-perms should ensure that a difference in ...


1

I've always had .bashrc files on my user accounts and never had this problem until I tried today to rsync something to my server using the root account. Your post helped me find the solution: my $user/.bashrc files always start with the following section to prevent this kind of issue. I replicated it to root's .bashrc and rsync'ing now works like a charm! ...


2

Don't use the -v option of rsync, instead use --out-format: rsync --out-format="%n%L" -az user@r18:~/assets ~/18 >> ~/18/rsync.log 2>&1 Will only have output when you are transfering files, and no output when you don't. To have more fancy output look at the rsync.conf man page, in the "log format" section.


1

There is an option called --link-dest which does what you need. When this parameter points to your latest backup it will only copy files from src to dest if src is different from the data already in your last backup (referenced by --link-dest) ; if the data is the same it will create a symlink instead thereby giving you a whole versioned backup. I use this ...


3

I used to do just this a few years back. (edit: with VMWare running on CentOS hosts, not ESXi admittedly) Every night I had a script that would suspend a VM, rsync the files from disk to the backup server and then start the VMs again. It worked quite well except... Rsync doesn't work very well with a 2GB file. Its not because rsync isn't brilliant, it ...


22

Because the transfer speeds out of the ESXi console are purposefully limited. Because this isn't scalable in any way. Because you'd have to drop a statically-compiled rsync binary onto the ESXi host. Because the VMs, the VMDKs, their ramdisk files and other components can change enough to make rsync a losing proposition... do you really want to re-sync a ...


8

Yes. Starting with rsync version 3.1.0 the --info=progress2 argument will give you progress on the entire transfer, including speed of the entire transfer. You can see a little bit of detail on the rsync man page.


0

Ok my problem was that I am "SSHing" on non standard port and I was trying to tell rsync via --port= option. I used rsync -e "ssh -p 1337" user@host:somepath/ . instead and it works just fine out of the box.


1

This is a problem with special characters in the file names, as discussed here: http://superuser.com/questions/91967/rsync-character-set-problems


0

Maybe you also want to take a look at http://backuppc.sourceforge.net/ It's more than just a script, it's a server side solution for backing up multiple servers. Maybe a bit too much for you, but also very popular.


4

Someone has already written a script that does this, and it has a lot of great features, plus it has been tested by many people for years. It is called "rsnapshot". I highly recommend "rsnapshot". It is easy to configure and will do a 3-day retention, or even more complex retention policies. http://www.rsnapshot.org/


0

I "solved" this by doing it the way I sort of didn't want to do: I added the following line to my rsyncd.conf: exclude from = /cygdrive/c/cygwin/etc/rsyncd.exclude And rsyncd.exclude is: - /dev/* - /dev - /cygdrive/* - /cygdrive - /proc Now it works as intended, with the caveat that this means I can't have directories / files named "dev", "proc" or ...


0

Turns out someone had actively gone and changed the logrotate configuration file to the adm group rather than the primary group. Therefore breaking the permissions for logaccess.


0

Are you using logrotate? Adm Is the group usually assigned by the log rotate daemon. See http://linuxers.org/howto/howto-use-logrotate-manage-log-files


0

Have you looked at rdiff-backup? It creates rotating backups kept for a certain number of days, and uses rsync as the transport method. It basically does everything you are trying to script automatically with no extra effort. It creates diffs for each backup, so if nothing has changed, no extra disk space is used. rdiff-backup backs up one directory ...


1

There are a couple ways to detect if --link-dest is working like you expect. One way would be be to use the find command to look for files that have hardlink count greater then 1. Something like find . -type f -links +1. The du command will also typically filter only count a single file once, even if there are many hard links to it. So If you were to use ...



Top 50 recent answers are included