Questions tagged [rsync]
Rsync is a fast and extraordinarily versatile file copying tool. It can copy locally, to/from another host over any remote shell, or to/from a remote rsync daemon
1,613
questions
0
votes
0
answers
17
views
Rsync to exfat external drive, quiet switch not working
I am attempting to use rsync to copy from my linux hard drive to an exfat external drive.
The working code to copy from ext4 to ext4 begins as: rsync -Razh
The code that I am attempting to use to copy ...
3
votes
5
answers
1k
views
Why is rsnapshot claiming source directory doesn't exist?
I'm trying to set up a backup system using rsnapshot which will copy files from a remote location into a local one. I've got rsnapshot set up, as far as I can see correctly, but when I run rsnapshot ...
- 623
0
votes
1
answer
60
views
How can I query the version number of a remote rsync server (daemon)?
How can I query the version number of a remote rsync server (or daemon)?
I can find out the version number of my local rsync client using --version:
% rsync --version
rsync version 3.2.7 protocol ...
- 71
1
vote
2
answers
4k
views
permission denied publickey using rsync with GCP
I'm trying to connect with rsync to a GCP vm. Whenever I do, I'm getting weird errors related to permissions.
I've checked that my username is part of the sudoers group so I can't really see where ...
CommunityBot
- 1
2
votes
3
answers
3k
views
rsync invocation to replace symlinks pointing to source?
Currently I'm moving a big filesystem to a new server as the original fileserver is no longer able to handle the filesystem writes.
To make this quick I made symlinks at the target filesystem ...
- 101
7
votes
3
answers
13k
views
Checksumming to verify rsync transfers
So rsync runs some checksums in the course of deciding what to transfer (i.e. what blocks within a file). But is there any reason to trust the file you end up with on the receive side any more than ...
- 103
0
votes
0
answers
92
views
sshpass/rsync fails when not redirecting stdout etc
I am having a weird issue and I just can't seem to find the cause and a solution.
I am on win11/cygwin (sshpass 1.06,rsync 3.2.7)
I call:
/usr/bin/sshpass -p $sshpasswd /usr/bin/rsync --dry-run -...
- 11
0
votes
0
answers
40
views
How to ignore folder modification time with rsync
When rsync compares src and dest trees, it checks for modification time. For folders, if the modification time changed, it'll update that modification in the destination. I'd prefer not. Ignoring the ...
- 1
0
votes
0
answers
34
views
Unreasonably slow SSH tunnel - but just in the opposite direction
While setting up rsync to a remote computer, I realized that syncing from my laptop the remote was fast, but the opposite was slow.
After some debugging and iperf3, I figured that this only happens ...
88
votes
12
answers
110k
views
protocol version mismatch -- is your shell clean?
When following the instructions to do rsync backups given here: http://troy.jdmz.net/rsync/index.html
I get the error "protocol version mismatch -- is your shell clean?"
I read somewhere ...
- 11
0
votes
0
answers
471
views
Ubuntu 20.04 / 22.04 - Remote ssh or rsync killing all process from user
I got a weird strange odd behavior (there's not enough adjectives for that) on Ubuntu 20.04 LTS and Ubuntu 22.04 LTS.
This is a IBM Db2 test server. The server runs a Db2 service under db2inst1 user ...
3
votes
2
answers
2k
views
passing additional parameters to rsync using rsnapshot
I have my backups being performed by rsnapshot, works well.
EXCEPT, on one of the jobs for the /home volume I'd like rsync to copy the sym-links as physical directory/files. I believe the parameter ...
- 1
0
votes
1
answer
30
views
Total space in bkps using rsync and hardlinks: who is right between du and Synlogy web interface?
I bkp my server root partition on a Synology NAS (fs: btrfs) mounted as NFS on /mnt/bkps using this script that is based on rsync and hard links, to to have daily bkps.
If I check the space using du ...
- 50.7k
0
votes
3
answers
991
views
rsync IMAP SENT folders between 2 users
How can I handle this:
We have a Email system based on DOVECOT IMAP, different users, different logins.
I want to somehow rsync all sent folders of some users, so one user can check what another ...
0
votes
0
answers
51
views
Rsync inline on large files: is is possible to skip the initial checksums?
Trying to sync large files (a VM image) between machines with rsync single.large.img user@host:/destination/ --inplace --progress or rsync *.img user@host:/destination/ --inplace --progress, the ...
- 22.9k
0
votes
0
answers
112
views
Script to wake up and shutdown a remote backup server after rsync command
How can I script the wake up of a remote backup server (with Ubuntu Server 22.04), and after rsync command is complete, shutdown the server.
For example, the server would turn on every Friday at 12am ...
2
votes
3
answers
451
views
rsync is not preserving file timestamps with the 'a' option when writing to cifs mount
My Ubuntu 22.04 VM (on my synology nas) preserves file timestamps when writing to it's local filesystem, but not when writing to a cifs mount (on the nas). This causes it to re-grab the non-preserved ...
- 286
6
votes
1
answer
5k
views
rsync from remote ssh server with forwarded keys
Context
I am writing a synchronisation script to sync some web environments.
Ex: Update integration from production (mostly to fetch new database rows and uploaded files)
Some of my prerequisites are :...
- 340
0
votes
0
answers
32
views
I upgraded Debian 10 to Debian 11, then Debian 12. Why does previously used commands now say out of memory?
I upgraded two low end vps systems only used for mail and dns from Debian 10 to Debian 11 then to Debian 12. No issues during either upgrade for either system. I have a cron that rsync's the vmail ...
- 1
4
votes
4
answers
7k
views
rsync error: error in rsync protocol data stream (code 12)
Guys this is an error I am getting when I try sending files to a server. I am not sure what went wrong. Can anyone help me fix this? Thanks in advance.
$ rsync -avz ./ ssc:/webapp/
rsync: connection ...
- 2,055
7
votes
4
answers
6k
views
rsync with --hard-links freezes
I have a large directory called servers, which contains many hard-links made by rsnapshot. That means that the structure is more or less like:
./servers
./servers/daily.0
./servers/daily.0/file1
./...
- 1,202
-1
votes
1
answer
120
views
how to sync /home directory in Linux (centos)
How can I keep in sync my /home directory on two different systems?
I found something like unison, but it is no longer in the repositories of centos9.
Are there other possible solutions?
The ...
- 2,594
0
votes
2
answers
6k
views
rsync error on Downloading
I am trying to setup a Rsyncmirror on my mount drive /mnt/Mirror/ubuntu_mirror, so I am trying the command:
rsync -a --bwlimit=128 rsync://archive.ubuntu.com/ubuntu /mnt/Mirror/ubuntu_mirror
but it ...
CommunityBot
- 1
0
votes
1
answer
86
views
Extract hard linked backup data from file server with rsync to exFAT file system
I need to gradually extract all data of the specific directory from Time Machine backup so:
it fits the destination drive size
it has no hard links because destination volume file system is exFAT
it'...
- 33.3k
2
votes
2
answers
233
views
How to make ddrescue rescue unreadable files along with rsync run when you backup files, when they exist only at the source?
How to make ddrescue rescue unreadable files along with rsync run when you backup files, when they do exist only at the source?
I think that major programs that work on backup disks must "merge&...
- 111
0
votes
0
answers
121
views
Combining rsync -t and --ignore-existing
I'd like to preserve the time of modification for files that did not exist on target, but do not change files that already exist, even if the modification times differ. I thought that rsync with -t ...
- 15
9
votes
13
answers
62k
views
rsync error: error in rsync protocol data stream (code 12) at io.c(600) [sender=3.0.6]
Recently I have been unable to rsync over ssh. Each time I get the same error
bash: rsync: command not found
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: ...
20
votes
8
answers
52k
views
Faster rsync of huge directory which was not changed
We use rsync to backup servers.
Unfortunately the network to some servers is slow.
It takes up to five minutes for rsync to detect, that nothing has changed in huge directories. These huge directory ...
- 3,843
0
votes
1
answer
140
views
How do I configure multicast to be used with rsync in batch mode
The rsync man page says "Multicast transport protocols can be used to transfer the batch update files in parallel to many hosts at once, instead of sending the same data to every host ...
- 12.8k
44
votes
4
answers
72k
views
How to output a list of changed files from rsync?
I am using rsync in a bash script to keep files in sync between a few servers and a NAS. One issue I have run into is trying to generate a list of the files that have changed from the during the rsync....
- 283
0
votes
0
answers
220
views
File transfers very slow in one direction
I am having trouble debugging a connection issue :
Server A (OMV) has a fast internet connection : 900mbps down / 600 mbps up
Client B (Windows 10 using wsl) has a decent internet connection : ...
- 1
34
votes
7
answers
34k
views
How can I use rsync with a FAT file system?
I would like to write a simple backup script that saves some data to a FAT drive. Should I reformat the drive and use a better file system or is it possible to use rsync with FAT? If so, what problems ...
0
votes
1
answer
57
views
Rsync creating infinite nested directories
To be truthful, I don't really know if this is caused by symlinks or rsync but I've stopped using symlinks and migrated over to using rsync months ago. So I guess this is just to be sure %100.
This is ...
- 301
38
votes
3
answers
47k
views
SCP transfer only modified files
I'm using below command to transfer files cross server
scp -rc blowfish /source/directory/* [email protected]:/destination/directory
Is there a way to transfer only files modified files just like ...
- 337
58
votes
5
answers
55k
views
Can I make rsync output only the summary?
I use rsync to backup a directory which is very big, containing many sub-directories and files, so I don't want to see the "incremental file list". I just want to know the summary in the end. If I use ...
2
votes
2
answers
210
views
rsync uncompressed to single compressed archive
Question
How can I rsync uncompressed (but highly compressible) data to a single compressed archive (.tar.gz, .zip, etc) without transferring all the data each time?
Current
This is what I am using ...
- 1,720
0
votes
1
answer
201
views
rsync cache/deduplication for similar files
I am using rsync to transfer/update 100s of TB from one fileserver to another, millions of files.
Many of those files are quite similar (but not identical).
I was wondering if there's a way to benefit ...
- 325
0
votes
2
answers
4k
views
Disable linux cache memory
Currently i have a server with 4 gb of ram runing 2 rsyncs task (100gb each) every 5 minutes and some monitoring containers, eventually the linux cache eats all ram (rsync faults) and if the ...
2
votes
0
answers
1k
views
Why on earth would rsync over SMB run nearly five times faster than rsync-to-rsync (over SSH)?
I just modified many files on a 36TB Unraid array (not quite half full yet). Now I want to sync all of those changes up to a backup array using rsync.
My usual rsync backup was going a lot more slowly ...
- 129
0
votes
1
answer
242
views
rsync cannot write files to folders it created
As the title says, Rsync is not able to copy files to the folders that Rsync itself created seconds before. I thought it could be a problem with curlftpfs, which is what mounts the ftp folder.
I ...
- 1,720
0
votes
0
answers
173
views
Rsync stopped syncing files
I recently moved to a new server and my rsync script that ran perfect for months stopped working. I'm at a loss for why. I have a 'dev' server and production server. When I run the following ...
- 101
4
votes
5
answers
17k
views
Can I rsync from Putty on windows to my Cent OS server?
I'm a bit new to the command line and UNIX but am finding it hugely useful. Is it possible for me to rsync from putty on my windows 7 machine to my virtual server?
If not what are my other command ...
- 111
3
votes
3
answers
7k
views
rsync link-destination remote server
I have a folder on a local machine and want to rsync to a folder on a remote machine. I would like to evaluate the copy against a folder on the remote machine --link-dest. The target directory will ...
- 21
0
votes
1
answer
42
views
copy many files with big sizes over rsync
I have database server on AWS and the backup of databases are over 1000 files with sizes can reach 5GB,
I am using a script to copy the backup to my data center using rsync, but the copy is slow, I ...
- 22.4k
20
votes
4
answers
25k
views
rsync --compress-level: which compression levels can be used?
Rsync has command line arguments for compression:
-z, --compress compress file data during the transfer
--compress-level=NUM explicitly set compression level
What does --compress-...
- 103
0
votes
1
answer
202
views
Suddenly, outbound scp, sftp, rsync, & ssh /bin/true fail; ssh still works
Problem
On my MacBook Pro running Mac OS 10.15.7, I cannot do outbound scp, sftp, or rsync to seemingly any server (I've tried ~10 I own). I also cannot run remote commands by ssh, e.g. ssh ...
1
vote
1
answer
184
views
Why is there a discrepancy between Finder and rsync when handling dot files?
Starting from two files
~/foo/bar1.txt
~/foo/bar2.txt
on my Mac's root volume, I copy the folder ~/foo (it's large; this is just an example) to /Volumes/Kiwi. I know from past experience that Finder ...
- 11.4k
16
votes
1
answer
19k
views
rsync: getcwd(): No such file or directory (2)
I want to sync Server1 and Server2's logs to LogServer.
For Server1:
rsync -avz -e 'ssh -p 2188' user@server1:/usr/local/servers/logs/* /usr/local/logs/
This one works, but for Server2:
rsync -avz ...
- 309
38
votes
4
answers
53k
views
How to repeatedly call rsync until files are sucessfully transferred
I'm trying to syncronize files from a remote server that is not reliable, meaning the connection tends to fail "randomly" with
rsync: connection unexpectedly closed
Rsync is called with --...
- 22.4k
3
votes
1
answer
1k
views
Rsync daemon, if it isn't using ssh, how does it authenticate the user trying to transfer data?
I understand that you can avoid ssh authentication with rsync daemon, but what method of authentication is it using then? How is it transferring the data?
- 80.3k