Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have 5 or so webservers at a loadbalancer and if a user uploads a pic it can take up to a minute to rsync it accross (crontab). This can get frustrating for some thinking it failed. Is there a way to do a real-time rsync?

Thanks

share|improve this question

6 Answers 6

up vote 11 down vote accepted

Most people design this kind of system using centralised storage, requiring no syncing at all.

share|improve this answer

As Chopper3 said, there's centralized storage using a SAN or NAS, and that seems to be the most common. If you really want to keep them on separate servers, you could look at something like DRBD, which is like RAID over ethernet, to keep a volume synced across servers.

share|improve this answer
    
Depending on your situation, you could try building a NAS-like setup with multiple commodity servers using DRBD and then use your web servers as a front end mounting the volumes on the servers as storage. Dedicated network connections could give redundancy as yes, NAS/SAN solutions are common for a backend storage, but it's also a single point of failure. Depends on your budget, needs and expertise. –  Bart Silverstrim Feb 8 '10 at 13:09

inosync is pretty much ready-made for this, provided the uploads arrive on a Linux machine (mac and bsd have different mechanisms and require a different utility)

share|improve this answer

You could also use inotify to watch a directory and instantly trigger propagation of file changes either via scp or rsync or any other method.

share|improve this answer

Historically you would used a shared network filesystem (canonical eg. NFS) to solve this.

These days I use MongoDB's gridfs for this type of multi-access shared object storage.

share|improve this answer

Right now we are doing some tests with inotify. I recommend these options too:

jnotify: File system events library for Java inotify-java: library provides bindings and an API for monitoring the filesystem on Linux platforms And of course: inosync

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.