Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Does anyone know of a bi directional mirroring tool to sync 2 different directories? Here is the scenario:

NFS mount under /data/nfs Document root under /data/www

Obviously we don't want to use an NFS mount over globally separated servers due to it's slowness and other issues. So the idea is to create a NFS mount in one directory that is hosted on a single file server and have the document root run on fast internal server drives. When a change is made on the document root side it syncs it back to the nfs directory and also if a change is detected in the nfs side it is synced over to the document root directory.

So why not use something like dropbox to sync? There is max file limitations and obviously cost involved in running cloud based servers including privacy issues.

I've looked at mirrordir but it seems to be one directional and not really suited to be run as a daemon as there can be some really bad consequences with deletes if they run at the wrong times.

So ideally it should be something along the lines of an inode notify makes a modification between the directories. I'm at wits end trying to find a solution for this without having to actually build my own solution but if it has to come to that then so be it. Just trying to find a better solution that doesn't need to be built from scratch.

I've also looked at using rsync / unison although it's error prone and slow when trying to sync bi directional.

share|improve this question
    
You've not mentioned the volume and frequency of data changes. This will have a huge impact on the solution you select. I've used NSFv4 over remote distances (private networks) with good effect but there was a low write volume and due to various cache mechanisms reads were quick. –  jeffatrackaid Jun 13 '13 at 16:05
    
We have a huge amount of writes as some cached files are written locally to the filesystem where there is no memcached available which is why NFS mounts are not an option, the other issue with a mount is potential issue with network if the network is down then so is the filesystem which is very problematic so a local sync between 2 directories is much better. I'm actually starting to work on a c++ app to do just that, do checksums between files and look at inotify for changes. If the one directory becomes fully empty notify of the issue if needed etc. –  Will H Jun 13 '13 at 17:22
    
That way if for instance the NFS mount goes down the daemon doesn't just wipe the entire doc root since the path is empty it waits for it to come back or notifies the owner of outage and just idles until fixed. –  Will H Jun 13 '13 at 17:23
    
I was a little confused by your initial post but if I understand correctly. You are creating an asynchronous replication system where the local directory will be synchronized to the NFS share on a periodic basis. The purpose of this is to mitigate network latency of long-ranged NFS mount and potential outages when these links are down. It may be redundant, but I would look into the FS-Cache system as it was designed to mitigate some of these issues. Also, I don't think it would cover hard failures. –  jeffatrackaid Jun 13 '13 at 18:53
    
Thanks @jeffatrackaid I'll take a look at it. I still think there is some use into what my app will eventually do. And yes you are correct on what I'm trying to do with the filesystem replication. –  Will H Jun 13 '13 at 20:09

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.