Questions tagged [rsync]
Rsync is a fast and extraordinarily versatile file copying tool. It can copy locally, to/from another host over any remote shell, or to/from a remote rsync daemon
1,613
questions
3
votes
0
answers
798
views
Getting lsetxattr errors using rsync with some files that have Apple resource forks
I'm running a backup of a MacOS machine to a Linux server using rsync. There are a few files that give errors every time, and I'm trying to figure out how to deal with them. I'd like to not get these ...
0
votes
1
answer
1k
views
rsync delete all source files that are synchronized with destination
I'm using rsync --remove-source-files --ignore-existing src/ me@server:dest/ to upload new files to the remote server without clobbering/updating any existing files. Any newly uploaded files are ...
0
votes
1
answer
628
views
Is it a bad idea to use rsync as root to sync /etc/letsencrypt and /etc/nginx between servers?
I have a failover setup with two servers and would like to sync /etc/letsencrypt and /etc/nginx between primary and replica.
I am planning on using Match Address on replica to only set PermitRootLogin ...
1
vote
0
answers
288
views
Compare 2 Directories and Move Differences to a Third directory
Running Ubuntu 20.04 and want to compare two directories called 1 and 2 and move differences to a separate directory called 3, ideally conserving the subdirectories.
I am able to get almost the ...
0
votes
1
answer
2k
views
Rsync Files only if they have not been changed in the last 2 minutes
I'm trying to write a simple cronjob that looks in a specific directory, performs a find command to determine if a file has been modified in the last two minutes, and if not, transfer that file to ...
0
votes
3
answers
4k
views
How to estimate how long an rsync of 427GB directory comprises of 470k subdirectories and 4476k files would take?
I have a directory that I want to rsync over a 1Gb LAN to another machine.
The rsync command is
/usr/bin/rsync -a --delete --numeric-ids --relative --delete-excluded --rsh=/usr/bin/ssh root@machine....
0
votes
1
answer
2k
views
Add timestamp to a filename during rsync
I'm using rsync to move files uploaded on an extern system to a internal system. However I recently had someone upload two different files with the same name. Meaning they had a file from Month1 and ...
1
vote
0
answers
217
views
Sudo needed a second time when rsync files with ssh key
I use a command to transfer files with rsync and a ssh key:
sudo rsync -P -e 'ssh -i <absolute path to keyfile>' <file> user@<server>
I need the first sudo because the file I'm ...
2
votes
1
answer
1k
views
Fastest way to tranfer data over the network between two servers
I have two storage servers at home - one is my workstation, which has a pair of 3TB disks in RAID1.
At first, I stored all of my data there. Recently, I received a small SBC with four 1TB disks.
I set ...
0
votes
0
answers
46
views
Backing-up Linux server's 700GB data comprising mainly of files less than 10KB
What would you recommend as backup for the following scenario?
I have a CentOS 7 Linux server, with about 700GB data, comprising mainly of files less than 10KB.
The backup media needs eventually to go ...
1
vote
2
answers
4k
views
Cygwin and rsync on Windows 10
I'm trying to use rsync in a Cygwin terminal to sync some folders with my linux server. The problem is, the password prompt comes up ok but when I enter the password, it is echoed to the screen and ...
0
votes
0
answers
184
views
Rescuing a linux server that cannot 'find' or execute any commands
Through a series of unfortunate events, a badly-configured script was executed with elevated privileges, which then executed the following command:
/usr/bin/rsync --recursive --links --copy-unsafe-...
0
votes
0
answers
666
views
rsync broken pipe between two servers on different networks
I have two FreeBSD servers, both on different networks, with a firewall in between. I have no control over the firewall.
Server 1:
IP: 10.1.0.2
GW (firewall): 10.1.0.1
Server2:
IP: ...
5
votes
1
answer
748
views
Bash: Dynamically traffic shaping rsync using variables for times schedule and slowing during runtime?
What are some solutions to having dynamic traffic shaping run during a bash script, after a command has already started? Is this even possible?
My use case is, I have rsync running on a huge set of ...
0
votes
0
answers
230
views
Linux ext4 incremental backup tool produces individual files not one
I want to backup ext4 files with size (<100MB) as they are, NOT in one single archive file (like with dump or tar).
Can you recommend me a tool that keeps the directories/files structure in the ...
0
votes
1
answer
432
views
Sync load balanced AWS Lightsail with multiple instances
I am planning to load balance AWS Lightsail with multiple instances.
Is it possible to auto sync files in main instance to another sub instances?
I would like to make easier to update themes and ...
-2
votes
1
answer
199
views
How to move a website to a new server? [closed]
I am going to move my website to a new server.
I need to copy the /var/www directory to the new server.
Should I use SCP or RSYNC ?
0
votes
1
answer
1k
views
rsync: Don't create destination directory
I want to rsync my root directory to an external hard disk mounted under /media/guettli/backup1t/.
rsync -xavi --delete --delete-excluded \
--exclude '...' \
/ /media/guettli/backup1t/
Something ...
-1
votes
1
answer
375
views
Rsync on synology error [closed]
I have created an SSH key shared between my Synology NAS and the Debian cloud server. I created this script to synchronize two folders, but it doesn't seem to work.
Command:
rsync -av --delete -e &...
0
votes
1
answer
178
views
How to syncronize XML files and store them as .gz at destination using rsync
I'm using rsync -z to backup a large amount of XML files. The source is a huge machine with thousands of GBs available on disk, but the destination is not that powerful.
What I want to do is to ...
0
votes
1
answer
294
views
Barman: rsync wal archival stalls on some files
I'm archiving wal files from a postgres server via rsync, most of the time the archiving works fine and is quick, the speedtest for the connection is here: (this goes via the internet)
[ ID] Interval ...
0
votes
1
answer
1k
views
rsync: rename File exists (17) on Fedora from a mounted Drobo CIFS share
I'm running a daily cronjob, using rsync to copy files from a Drobo FS, which is mounted locally via autofs on demand as a CIFS share. Why do I get these warnings about the file existing and a rename? ...
1
vote
0
answers
942
views
rsnapshot / rsync protocol version mismatch -- is your shell clean?
on both computer same version of rsync:
rsync --version
rsync version 3.1.2 protocol version 31
cat /etc/rsnapshot.conf
[...]
backup root@IP:/etc/ localhost/ ssh_args=-i /root/.ssh/id_rsa ...
0
votes
1
answer
408
views
How can I rsync a directory that might be busy with updating files?
I would like to transfer a directory of files to a remote destination. Easy.
However, at any given time, one or many of these files may be in-use and being updated by users.
This doesn't bother rsync ...
1
vote
1
answer
4k
views
Where can I find log of an rsync command run on the remote machine?
I'm running rsync via ssh to a remote machine. After running the command, I'd like to confirm that the command I sent to the remote machine is the one that actually ran. Is this logged somewhere? The ...
0
votes
0
answers
154
views
Unable to preserve permissions when rsync'ing files owned by root:root to a macos
I have files on a linux server with ownership permission of root:root. When rsyncing to my local mac, these files change ownership permissions to root:wheel because there is no root group on a mac. I ...
1
vote
1
answer
646
views
Low performance on file copy operations due large idle times (rsync, cp, ..)
I'm currently copying large directory tree from an external HDD to to internal storage of my server. The performance of the copy operation is basically quite good, i.e. 35MB/s with the external HDD ...
0
votes
2
answers
631
views
Rsync not working over OpenVPN
Discrete Question: Is there a problem with my network configuration or rsync configuration preventing backups over OpenVPN between two servers?
I have a working openvpn tunnel I can ping on 10.8.0.1. ...
0
votes
1
answer
576
views
rsync /home structure but only specific subdirectories [closed]
I need to copy / rsync the whole /home directory but only certain subdirectories.
Sample:
/home/100/someuser/Documents <-- copy with subdirs
/home/100/someuser/scans <--- copy with subdirs
/...
1
vote
1
answer
799
views
Using rsync --remove-source-files & --include-from= options together
I have a central CentOS 7 backup server, where I run the following rsync command in a script to pull the Application logs from 100's of Linux machines and then delete them.
rsync --remove-source-...
2
votes
1
answer
694
views
Rsync using absolute paths
I have two RHEL 7 servers.
On my Source I have the following directories
/sourcestuff/testresync
/sourcestuff/testresync/incl1
/sourcestuff/testresync/incl1/test1.txt
/sourcestuff/testresync/incl2
/...
0
votes
1
answer
366
views
OpenShift rsync error
I am developing a python3 script which replicates data from PersistentVolumeClaims mounted in a pod, via rsync. This script is executed in a pod, therefore, the rsync connection is from a pod to ...
0
votes
2
answers
797
views
"chroot", but with a VM?
I just backed up a remote system using rsync. I have a directory that I can chroot into and would now like to boot it up as a VM. I know that qemu has -kernel capability, as well as -drive file=fat: ...
0
votes
1
answer
374
views
rsync creates an empty directory on some hosts
I am running a backup script as a cron-job that looks (roughly) like this:
# Generate temporary directory for output
dest_file=backup-$(date +"%Y%m%d-%H%M") # e.g. backup-20200417-0400
abs_dest_file=/...
1
vote
0
answers
30
views
restore centos vps with rsync [duplicate]
today morning I woke up and saw our MongoDB databases are removed and just this DB HOW_TO_RECOVERY_BASE is there. that have a collection which is README and here is content:
{
"_id": {
"$...
1
vote
1
answer
451
views
speeding up du (maybe use ncdu) for backups with hard links
du -hc --max-depth=1 /home/back/tvppclientarea/ | sort -k2
To show the size of backup directories that use rsync and hard links. The command lists each directory and shows the amount added to the ...
0
votes
0
answers
4k
views
rsync postgres backup to another directory
I have CentOS 7 (3.10.0-1062.18.1.el7.x86_64) with SELinux enabled.
PostgreSQL 12 installed (not sure if it is related).
I try to backup database and move backup files to another disk. How I create ...
0
votes
0
answers
94
views
how to config rsync as deamon when a file change and not with cron
I have multiple folders that I rsync now with cron like in this example.
rsync -aqz source1/ user1@machine1:/some/destination1
rsync -aqz source2/ user2@machine2:/some/destination2
... and so on
...
2
votes
2
answers
752
views
rsync command works in terminal, fails in rundeck
I'm trying to use rundeck to rsync files in a docker container volume to an offsite server. I can impersonate the rundeck user and run the rsync command in a terminal window just fine.
When rundeck ...
1
vote
0
answers
2k
views
rsync failed to set permissions: operation not permitted
I am trying to replicate my domains group policy from my primary domain controller to my secondary domain controller. When I run the below command which I used from techmint https://www.tecmint.com/...
2
votes
1
answer
2k
views
rsync: Only update owner/group and permissions?
I'm in the process of duplicating a remote webserver and am using rsync to transfer the files:
rsync -rvz --ignore-existing -e ssh . root@(localip):/usr/files
It took about 12 hours to transfer ...
1
vote
1
answer
479
views
Rsync files to a server while changing their ownership without root access
I'm trying to allow users to rsync directories to a remote server, while changing the user and group of the uploaded files/directories to nobody:nogroup.
Each user has SSH access to the server via ...
3
votes
1
answer
880
views
How rsnapshot manages moved and/or renamed files and folders
I consider using rsnapshot for backuping large amount of data and I wonder how rsnapshot manage when a folder at the source is moved or rename, is it detected it and edit associated hard links or it ...
0
votes
1
answer
263
views
How do I rsync distant files with restricted permissions?
I tried to rsync files through ssh with a distant 'user' that does not have the permissions to even read them.
For instance file.conf which is:
root:root rwxr-----
What is best practice for ...
1
vote
1
answer
293
views
Allow rsync to write to httpd_user_content_t
We have rsyncd running as a daemon on our frontend web servers. We push web app updates out via rsync.
We're looking at cutting down the noise in auditd and fixing up some of the issues from SELinux (...
0
votes
1
answer
690
views
Rsync complex number pattern matching
I am working on migrating a large WordPress website migration and only want to migrate original images. I do not want to copy the images the WordPress creates. Using rsync, I am not seeing a way in ...
0
votes
1
answer
441
views
Tar directories while sending with SSH
Nowadays, when I want to backup my server, I log in to it and then I execute the command below which tars the local directory "/ebs1" and sends it to another server at IP xxx.xxx.xxx.xxx and saves it ...
0
votes
1
answer
376
views
What is a recommended way of thransferring am iocage jail from one host to another
Source iocage thick jail running on FreeBSD-12.0p12
Destination FreeBSD-12.1p1
Both hosts are using zfs.
What is the recommended method to transfer an existing iocage thick jail from the source to ...
6
votes
1
answer
7k
views
Does AWS cli do a data integrity check on sync with s3? [closed]
I've been looking into using AWS cli for data integrity checks to verify a backup has been transferred from a Linux file server correctly to AWS s3. Likewise, I would like to verify when restoring a ...
0
votes
0
answers
133
views
Moving files based on size with ascending order from one server to another in linux
How to move the files based on the size with ascending order from one server to another in Linux.