Skip to main content
15 votes
Accepted

Does rsync change file content (dealing with hardlinks)?

With standard options, rsync will copy the new file assigning it a semi-random name, then it will rename the new file with the original name. In this process, no writes are directed at the original ...
shodanshok's user avatar
  • 51.2k
14 votes
Accepted

Access Denied when syncing between s3 buckets on different AWS accounts

The cause of your ListObjects error is that you assigned permission to access the contents of your bucket (arn:aws:s3:::bucket/*) but you did not give permissions to the bucket itself (arn:aws:s3:::...
John Rotenstein's user avatar
8 votes

How to interrupt software raid resync?

Possible solution for this, took a bit to get into the details. My system: CentOS 6.5 mdadm v3.3.2 Constant checks every week, wanted to pause one of them, RAID is clean, check was called via the /...
bill.rookard's user avatar
7 votes
Accepted

Problems with Azure AD connect custom install

Azure AD connect custom installer is a tough enemy to fight with. And as you truly mentioned the errors is not clear and from time to time I have a filing that MS just don't want to waste a code on ...
Strepsils's user avatar
  • 5,135
7 votes

Sync Servers with DRBD

As it was mentioned already, DRBD isn't greatest solution especially if you have a requirements for further scale. I believe that the best solution on Linux would be Ceph Cluster - https://en....
batistuta09's user avatar
  • 9,200
6 votes
Accepted

unison-fsmonitor doesn't detect any changes

The executable unison-fsmonitor is not meant to be run on its own, but is instead just a tool that Unison uses to watch your filesystem. To have Unison watch your filesystem for changes just run ...
Mike Pierce's user avatar
5 votes

Live file syncronization across multiple Linux servers with millions of files and directories

I'd seriously recommend using something like ZFS for the filesystem. Built-in tools like the ZFS snapshot and ZFS send/receive allow you to take block-level snaps of the filesystem and ship it to a ...
ewwhite's user avatar
  • 199k
5 votes

Faster rsync of huge directory which was not changed

You can also try lsyncd, which will rsync only when changes are detected on the filesystem and just the changed subdirectories. I've been using it for directories with up to two million files on a ...
Juanga Covas's user avatar
5 votes

Sync Servers with DRBD

Yes, you can absolutely do that! See: https://docs.linbit.com/doc/users-guide-83/s-three-nodes/ The only problem you need to have cluster-aware file system or do active-passive-passive mode with ...
BaronSamedi1958's user avatar
4 votes

Syncing user accounts across multiple servers

For smaller/simpler deployments, a project called Internet Account Replication (IAR) could help. It has a simple server-client architecture and synchronises the credentials using SSH.
Richlv's user avatar
  • 2,393
4 votes

Rsync : copying over timestamps only

Use the source (e.g. /path/to/source) directory as reference for the touch command. Just cd to your target directory and do find -type f -exec touch -r /path/to/source/{} {} \; You cannot copy sub-...
knorke's user avatar
  • 41
4 votes
Accepted

Rsync over sshfs skips filenames starting with underscore

It turns out the culprit was in the sshfs flags. The noappledouble flag I was using to get rid of .DS_Store files was actually interfering with rsync's work. From the sshfs Mount Options docs: ...
Nic Nilov's user avatar
  • 191
4 votes

How to upload a directory recursively to an FTP server by just using ftp or lftp?

cd {local_dir} lftp {server} cd {remote_dir} mput {local_dir}/* This worked for me, many other attempts were failing. Once in lftp, more info available via: help mput
austin's user avatar
  • 141
4 votes
Accepted

If two scheduled tasks are missed, in which order will they run next?

The order is not specified, it's an implementation detail. So you can't rely on the execution order, because it can be random, or change with Windows updates. You have two options here: Disable Run ...
Swisstone's user avatar
  • 6,945
4 votes
Accepted

Can I determine how much CPU is used for calculating RAID parity?

Open several terminals and keep an eye on top or htop. Look specifically for spikes in CPU usage during database commits. top or htop Use iostat to see how much time the CPU spends in I/O operations:...
Max Haase's user avatar
  • 847
3 votes

Is it possible to change an MIT Kerberos password programmatically?

When a user changes their Active Directory password, I would like to sync it back into MIT Kerberos. In its current form, this is too vague of a statement for it to really be answered in one shot. ...
Ryan Ries's user avatar
  • 56k
3 votes

Live file syncronization across multiple Linux servers with millions of files and directories

If you cannot change the filesystem on the production server, I would put the files on another server and mount them with NFS. I would use Linux and ZFS if man-hours are inexpensive, maybe some kind ...
Law29's user avatar
  • 3,577
3 votes

Faster rsync of huge directory which was not changed

I think that 80k files is nothing extraordinary today. My explanation for the problem lies in the way how rsync works: See here. They say: While it is being built, each entry is transmitted to the ...
Petas's user avatar
  • 41
3 votes
Accepted

chrony does not log anything on my ec2 instance

I just verified chrony logging by doing the following: created a default Ubuntu 18.04 LTS EC2 instance - AMI id: ubuntu/images/hvm-ssd/ubuntu-bionic-18.04-amd64-server-20180912 (ami-07a3bd4944eb120a0)...
Paul Gear's user avatar
  • 4,622
3 votes

Best way to transfer about 200GB of files between two windows servers

Robocopy is good for moving a lot of data. Command help here - https://ss64.com/nt/robocopy.html
Edwrd_T_Justice's user avatar
3 votes

Best way to transfer about 200GB of files between two windows servers

Is downtime acceptable at all? Sneakernet is still a thing, and depending on the physical distance it may provide enormous bandwidth. For machines with limited allowed downtime it might still be an ...
Mikael H's user avatar
  • 5,149
3 votes

rsync: Remember copied files, even if they are deleted at the destination

Two different approaches are possible: make a selection of the files that need transferring and feed that list to rsync and only those files will be copied. The find command is particularly useful ...
Nobody's user avatar
  • 31
2 votes

how i can disable delete file option in unison sync tool?

There's now a -nodeletion option that prevents unison from deleting files on one of the roots, or to both roots if you need that.
BxlSofty's user avatar
  • 753
2 votes

How to have samba password changed from linux ldap client?

The way to enable this has changed over the last ten years, so I've written up a post that explains how to enable smbk5pwd in simple terms here. Short version: Install ldap contrib modules, eg for ...
Myk Dowling's user avatar
2 votes

Faster rsync of huge directory which was not changed

Use rsync in daemon mode at the server end to speed up the listing/checksum process: Rsync daemon: is it really useful? http://giantdorks.org/alain/achieve-faster-file-transfer-times-by-running-rsync-...
Gringo Suave's user avatar
2 votes

Faster rsync of huge directory which was not changed

For synchronisation of large numbers of files (where little has changed), it is also worth setting noatime on the source and destination partitions. This saves writing access times to the disk for ...
Andy Beverley's user avatar
2 votes

How can I *include* folders through Dropbox's selective sync feature in Linux, from the command line?

If you need to simply sync one deep directory you may need to repeat the previously accepted Answer procedure recursively. For example, if you are trying to sync a Minecraft server to a hosted VM, ...
r3cgm's user avatar
  • 327
2 votes
Accepted

How to keep newly installed packages in sync from master to nodes with puppet?

There is no way of doing this that I'm aware. I might suggest that you're approaching this with the wrong attitude somewhat. The idea behind Puppet is not to do a bunch of manual work on a single ...
jaxxstorm's user avatar
  • 626
2 votes

real time sync for file sharing servers

GlusterFS is a great for this, as is Ceph. GlusterFS is easier to manage, and does not use node-to-node replication as a primary method of data replication or distribution. It can perform 2n or 3n ...
Spooler's user avatar
  • 7,136

Only top scored, non community-wiki answers of a minimum length are eligible