4
votes
Discrepancy in file usage after rsync
coreutils du - even when called with --apparent-size (implied by -b) - may add a non-zero size of directories to the total. This size will be 512 bytes for an empty FAT directory, but 4096 bytes for ...
- 9,783
4
votes
Accepted
Test network transfer speeds with rsync from a server with limited storage
To test the raw network bandwidth, you can use iperf (or iperf3). You can use something as simple as:
on the server: iperf -s
in the client: iperf -c <server_ip>
then, reverse the server and ...
- 50.7k
4
votes
Accepted
rsync: No space left on device (28)... with a lot of free space
This error seem to come from attempting to sync xattrs that are too large for the target filesystem.
I got the error by syncing files from btrfs to ext4, a directory had a rather large value in the ...
- 156
3
votes
Filenames change when RSYNC to NAS with CIFS or NFS
It seems that my thoughts I was mounted with NFS was not accurate. Once I was able to properly mounted the NAS with NFS, I was able to RSYNC and maintain the desired file name. My issue was with the ...
- 61
3
votes
Rsync daemon, if it isn't using ssh, how does it authenticate the user trying to transfer data?
The rsync daemon, without SSH, provides anonymous access and/or it's own username/password authentication methods as part of the protocol.
On the server side you configure a plain text file with the ...
- 80.3k
2
votes
rsync directory is smaller at destination
The reason for the larger size locally is the local drive has a block size of 128KB, where the remote machine's block size is 4KB. Despite file sizes being shown in bytes, a file must occupy an ...
- 1,893
2
votes
Rsync when system resource utilization is low
It depends on the resource, but you can do the following:
CPU: set the nice level of the process to a high value (lower it's
priority)
Disk: set the ionice level of the process to an idle
scheduling ...
- 1,782
2
votes
Why is there a discrepancy between Finder and rsync when handling dot files?
Short answer: Don't worry about it. Those aren't actual files being copied; it's just an artifact of how Apple's version of rsync handles complex file metadata over a protocol (rsync v2) that doesn't ...
- 11.4k
2
votes
Accepted
How do I configure multicast to be used with rsync in batch mode
rsync doesn't support multicast itself, it merely suggests multicast as a method to transfer the batch (delta) file:
BATCH MODE
Batch mode can be used to apply the same set of updates to many
...
- 12.8k
2
votes
How to make ddrescue rescue unreadable files along with rsync run when you backup files, when they exist only at the source?
If source media is unstable, address that problem (dump and replace it, or restore from backups, whatever) first, before doing any other operations, including regular backup, with rsync or anything ...
- 12.6k
1
vote
How can I query the version number of a remote rsync server (daemon)?
While I don't know of a built-in way to determine the version of a remote rsyncd daemon, you can get an idea by telnetting into port 873 to view its response. It will greet you with the protocol ...
- 81
1
vote
Accepted
Total space in bkps using rsync and hardlinks: who is right between du and Synlogy web interface?
du -hs is right. The Syslogy WebUI is probably not hardlink-aware.
You can double check this by monitoring the volume free space.
- 50.7k
1
vote
Rsync creating infinite nested directories
Follow the example below.
If I have src=/a/b/test path having the file test and another is dst=/c/d. If I need to rsync that file from src to destination.
I will use this command.
rsync -azsL /a/b/ /c/...
- 301
1
vote
rsync cache/deduplication for similar files
Answering myself: It looks like --fuzzy is just something similar:
This option tells rsync that it should look for a basis file for any destination file that is missing. The current algorithm looks in ...
- 325
1
vote
Accepted
rsync cannot write files to folders it created
The clue is here:
rsync: mkstemp "/mnt/ftp/.../daily.0/.backup_to_ftp.tar.lmLoNR" failed: Operation not supported (95)
It's telling you that the remote system - or one of the cludges used ...
- 1,720
1
vote
How to make ddrescue rescue unreadable files along with rsync run when you backup files, when they exist only at the source?
Disclaimer: if you are trying to recover something valuable use the right tool for the job. Using ddrescue to make a full image of the disk "before" attempting recovery is the right approach....
- 111
1
vote
Test network transfer speeds with rsync from a server with limited storage
I would use a network bandwidth monitoring tool instead, e.g. iperf3.
You can run the server instance of it on any of the hosts and from the client decide which direction bandwidth you like to ...
- 74
1
vote
Accepted
rsync skip transferring files that take longer than n minutes
I found out that this is what is timeout command intended to do. If you want to let the main command communicate with the terminal you need to pass --foreground switch.
timeout --foreground 300 rsync ....
- 135
1
vote
rsync is not preserving file timestamps with the 'a' option when writing to cifs mount
I just had this issue (rsync silently not preserving timestamps) when trying to copy ~60 GB of jpg/mp4 files from a src: exfat mount to a dst: cifs mount.
At first I wondered if it was the source ...
- 554
1
vote
Disable linux cache memory
vfs_cache_pressure is the appropriate sys parameter to tell the kernel how quickly you want it to discard cache.
Linux Performance: Almost Always Add Swap Space
1
vote
rsync error: error in rsync protocol data stream (code 12)
GentlemanJohn's answer was extremely helpful for me. I took his advice but I used /user/bin/ssh instead of revising my path. As in:
$ rsync -avz -e "/bin/ssh -i .ssh/id_rsa" --progress ads@...
- 11
1
vote
Faster rsync of huge directory which was not changed
Use ZFS and replicate using snapshots if this fits your use case.
Use a distributed filesystem if you need write access on both sides (ceph/gluster/microsoft dfs etc)
Rsync is nice for adhoc or small ...
- 369
1
vote
Faster rsync of huge directory which was not changed
I appreciate I'm 8 years late to the party here, but I found this thread while searching for a solution for a similar situation. I have a directory on my home LAN containing webcam videos. Typically ...
1
vote
How to make sshfs + VPN + git a tolerable working environment?
I recently use mutagen instead of sshfs it's much better when working on unstable connection.
https://github.com/mutagen-io/mutagen
mutagen sync create --name=test --default-owner-beta="USER_HERE&...
- 61
1
vote
protocol version mismatch -- is your shell clean?
I found another case, if the default shell of your server is zsh, then you need to check other files like .zshenv, .zprofile, .zshrc etc. Avoid them having "echo"
Use @Stabledog 's method ...
- 11
1
vote
How to repeatedly call rsync until files are sucessfully transferred
Just another take - a simple one liner to run indefinitely until succeeded, with a 1 second delay after a failure.
while true; do rsync -avz --info=progress2 --partial src/ dsr/ && break || ...
- 171
Only top scored, non community-wiki answers of a minimum length are eligible