Tag Info

Hot answers tagged

7

If you can trust the filesystem last-modified timestamps, you can speed things up by combining Rsync with the UNIX/Linux 'find' utility. 'find' can assemble a list of all files that show last-modified times within the past day, and then pipe ONLY that shortened list of files/directories to Rsync. This is much faster than having Rsync compare the metadata of ...


5

Try unison, it was specifically designed to solve this problem by keeping the change lists (building file list), locally to each server, speeding up the time to calculate the delta, and the reduce amount that is sent across the wire afterwards.


3

Many of your problems are handled by configuration management tools like Puppet, Chef or cfengine. It works differently than you envision it, but it's the goal that matters, right? With e.g. Puppet, you install the basic OS (might be from an image), with a Puppet client and then Puppet handles the installation/configuraton of the software. To a degree, ...


2

You should seriously consider DRBD. DRBD is a RAID1 over TCP-IP software. It will replicate a block device over any link in real time. Every time you modify a HD block, it will replicate it. It is FS-agnostic therefore you can put any FS you want on top of it. It will also work hand-in-hand with heartbeat which will allow you to start the hot spare as soon ...


2

http://oss.linbit.com/csync2/ is designed for this sort of thing, I'd give that a try.


2

How about DRBD with its dual-primary mode and a cluster file system such as OCFS2 or GFS? Might work surprisingly well especially if your directory tree is not containing a huge amount (say, millions or more) of frequently changing small files.


2

Unison is a 2-ways synchronization tool. If you only want to synchronize in one-way, take a look at --force, --prefer option or use rsync for alternative.


2

two easy answers, (and the same you've already mentioned): Unison if you want something that 'just works'. Has a nice GUI and you can easily define a task there and call from command line, a script, cron, whatever. I use it to synchronize both my laptop and my wife's with our respective user directories on the desktop. It's really that easy. rsync if ...


2

GlusterFS? You would have both web servers as gluster servers (say, exporting /srv/export), and also acting as gluster clients, mounting the same exported volume (say, moutning /srv/share). File operations would be on /srv/share and would propagate to all boxes in the cluster.


2

We use a wrapper around Unison that, on failure, creates a file in /tmp based on the pathname of the directory to sync. Then, there is a service check in Nagios that checks for the presence of these files, and sends up an alert if found. This way there is the verbose Unison log for more information, a quick way to know if something went wrong, and ...


2

I wouldn't use Unison in this case, since its main purpose is the two way sync. Plain old rsync will do the job here. This will synchronise files from the source to the destination without deleting files on the destination (although it will overwrite files on the destination to be the same as on the source). rsync --recursive --times --verbose source/ ...


2

There is no global config file, but a shortcut: Use include file to source other config files. Now you can save the above lines to common and add include common at the beginning of each of your config files. There is a more sophisticated setup in the official manual.


2

I solved the problem, and it was the HOME=/ in crontab, it should be the path where your .unison exists So my crontab content now is SHELL=/bin/bash PATH=/sbin:/bin:/usr/sbin:/usr/bin MAILTO=root HOME=/home/user1/ # For details see man 4 crontabs # Example of job definition: # .---------------- minute (0 - 59) # | .------------- hour (0 - 23) # | | ...


1

Why are you running unison as root? Why not use sudo -u bitnami unison instead? Then the file you mentioned will automatically belong to bitnami, and you have the added benefit of not exposing your server to accidental file deletions or other mishaps with root privileges.


1

Looks like you have to place the unison options after defining the root directories: Formatted like this: link to unison manual (RTFM!) unison root1 root2 [options] So my code should be: ${_unison} -batch "${p}" "ssh://${r}/${p}" -times -owner -group Once the options are placed here, the script executes without any errors.


1

I think GlusterFS can solve your problem. It is a distributed filesystem that operates in userspace, so you don't have to change your current filesystem. With a gluster replicated volume you have a single virtual FS, which you can mount on your servers using FUSE module or NFS. This virtual FS is immediately synced between the nodes on your cluster.


1

This asymmetric architectures doubles your chance of the service failing - and it's not very good for performance. i use rsync ( as explained above ). I didn't see an explanation of how you use rsync. In order to sync the application code Deployment of application code should be a managed process. How you manage replication of content uploaded ...


1

You can use tools rsync or unison, but you will not get a realtime sync. There should be some delay until new/updated files get synced even if you run such a tool every minute. You can implement it if you think it is acceptable to have a small delay. If you want to get changes synced in realtime, you need to use a tool like NFS/glusterfs, or DRBD. The main ...


1

I believe Unison works like rsync (as it uses rsync code) and therefore does not pick renamed files. If you change the name of a file on the source box unison will delete the old file and re-transfer the "new" file. The outcome of changing the name of a file on the destination box depends on how you setup unison. If you have it deleting files that don't ...


1

Why not to point Apache to your shared folder on VM in the first place? This is as simple as to add/change one file for your virtualhost (I believe you've done this anyway). This will let you not to wait 5-10 seconds while your server document root is being updated. However what you actually ask is a periodically running some command, so in the end you ...


1

You should be able to setup ssh login with keys as you have stated. Make sure your sshd_config allows root logins PermitRootLogin without-password Then copy the generated public-key to /root/.ssh/authorized_keys. You should be able to login now, just tell unison to use the ssh key you generated.


1

Whats wrong with the normal rsync output? /tmp # ls -l ra total 8 -rw-r--r-- 1 phemmer unix 5 2011/02/09-16:42:49 adsf -rw-r--r-- 1 phemmer unix 29 2011/02/09-16:49:41 date /tmp # date > ra/date /tmp # rsync -Haxv ra rb building file list ... done ra/date sent 180 bytes received 42 bytes 444.00 bytes/sec total size is 34 speedup is 0.15 Just ...


1

I would use a backup program like Bacula to do this. It keeps a database of your backups so it don't need to compare file sizes and time stamps against already backed up files like rsync. If you want to use rsync, you could use find to find files changed after your last backup, write this to a file and read the file names to copy from this file with ...


1

I critically need the sync to be immediate as the user needs to display the thumbnail straight away on a web page So you've no control over server affinity in your load balancing? Time to get a new load balancer. I looked at unison and lsync, but there doesn't seem to be a rpm available from the centOS5.5 repo for either. Previously I did this by ...


1

If you're using the -z switch on rsync, try running without it. For some reason I've seen this speed up even the initial enumeration of files.



Only top voted, non community-wiki answers of a minimum length are eligible