up vote 1 down vote favorite
share [g+] share [fb]

Folks, I have an "origin" server where data is being dumped using WebDav. I need to distribute this data to edge caches as soon as the data is received on the origin. I will use either Apache or Nginx for the origin and the edge server.

What is the recommended way to do a live mirroring of my content from origin to edge? Is it possible to program WebDAV triggers into Apache/Nginx that will result in the content being propagated, or should I look at something simple like rsync?

Thanks for your insights.

-Raj

link|improve this question
75% accept rate
Why do you need to 'push' data to NGINX? Would it not be easier just to let NGINX pull it but set/check ETag/last-modified headers? – sam Jun 21 '11 at 18:40
Hi Samarudge, thanks for your response. I am generating fast changing content that needs to be on my edge nodes before the clients get there. My data source publishes the content on one server using WebDAV and I need to rapidly mirror this content on edge nodes. – Raj Jun 21 '11 at 18:54
Makes sense, I'm not an expert on WebDAV (I use GIT) but in GIT you can attach events to the 'post-commit' hook that will be run on every commit. I'd suggest hooking an 'rsync' to that, but like you say in your question I'm not sure if that can be done. – sam Jun 21 '11 at 19:27
feedback

1 Answer

up vote 0 down vote accepted

inotify and rsync is a crude but effective method for doing this. Lsyncd does the job for me, but I'm ok with a few seconds lag between my origin and edge nodes. If you're after something quicker you'll need to consider spending a lot of time/money.

link|improve this answer
lsyncd looks like the thing I need. Frankly I am surprised rsync does not already do this. Keeping remote folders in sync on an ongoing basis would appear to be a pretty common use case. – Raj Jun 30 '11 at 22:04
feedback

Your Answer

 
or
required, but never shown
discard

Not the answer you're looking for? Browse other questions tagged or ask your own question.