Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have an app farm which generates thumbnails and makes them available to a web server farm using nfs. However the performance is so poor so I am going to make a local copy of the files on each web node.

I looked at unison and lsync, but there doesn't seem to be a rpm available from the centOS5.5 repo for either.

which one should I use, or some other strategy. I critically need the sync to be immediate as the user needs to display the thumbnail straight away on a web page, so inotify based tools are good!

Edit: when I say immediate, I mean pushed out rather than pulled on some schedule.

In the end, lsync have released some more versions so I used that, and lsync works well once it is running (ie syncs in a few seconds) however it needs to enumerate each directory beforehand and for a very large file-system takes many hours..

share|improve this question

3 Answers 3

I critically need the sync to be immediate as the user needs to display the thumbnail straight away on a web page

So you've no control over server affinity in your load balancing? Time to get a new load balancer.

I looked at unison and lsync, but there doesn't seem to be a rpm available from the centOS5.5 repo for either.

Previously I did this by building replication into the application - after all, it's the first thing to know when new content needs to be replicated - it was just a few lines of PHP to publish a notice to the other nodes that content needed fetching - when a node has been offline for a while, I ran rsync before bringing it completely back online (this approach makes handling offline nodes much simpler and non-intrusive). However if I were to implement such a system today then I do it using overlay filesystems - perhaps an NFS at the bottom of the stack and copy up on access.

See also this post

HTH

C.

share|improve this answer
    
well the idea thing would be to build something into the app, and preferably better design the nfs layout. but thats not possible at the moment –  Tom H Jul 9 '10 at 3:20

I don't see anything for it in the repository, but you might find that incron may be helpful.

share|improve this answer
    
hi, thanks for the reply. incron is not recursive. though it looks kinda cool and i will be messing with it for fun at some later date. –  Tom H Jul 9 '10 at 3:19

I recently installed unison version 2.32.52 on a one of my CentOS 5.x boxes.

With unison its important to note that all servers that will be synchronizing together will need to be running the exact same version. Unison won't work when linked to other unison versions older and newer.

For myself I found that this rpm http://rpmfind.net/linux/RPM/dag/redhat/el5/x86_64/unison-2.32.52-1.el5.rf.x86_64.html worked for me running on centos 5.8

As for lsync I wouldn't be able to tell you much about it as I have never personally used it. I can however say that Unison when used properly is a fantastic tool.

You can set Unison on a crontab to check for changes every 1-10 minutes.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.