I have the following configuration of servers:

  • 1 central linux server, a VPS
  • 8 satellite linux servers, "crappy shared hostings"

I have a bunch of files that I need to have in all servers. Right now i'm copying them everywhere manually, but I want to be able to copy them to the central server, and then have a scheduled process that runs every now and then and synchronizes them (only outwardly, no need to try to find "new" files in the satellite servers).

There are a couple of catches though:

  • I can't have any custom software in the satellite servers, or do strange command line things that'll auto connect to them and send the files directly. I know this is the way these kinds of things are normally done, but the satellite servers are crappy shared hosting ones where I have absolutely no control over anything. I need to send the files over FTP

  • I also need to have, in my central server, a list of the files that are available in each of the satellite servers, to make sure they are ready before I send traffic to them.

If I were to do this manually, the steps would be:

  1. get the list of files in a satellite server
  2. compare to my own, and send the files that are missing
  3. get the list of files again, and store it in my central database.

I'd like to know what tools are out there that can alleviate as much of this as possible, first the syncing, and then the "getting the list of files available in the other server".
I'm going to be doing everything from PHP, not sure if there are good tools to "use FTP from PHP", which i'm pretty sure i'll have to do for step 3 at least.

Thanks in advance for any ideas!

  • 1
    Do your hosts have rsync?
    – codeholic
    Apr 27 '10 at 0:06
  • @Daniel Are you looking to build the solution in PHP yourself, or are you looking for something (built on PHP) that you can drop in place on your server(s)?
    – Sampson
    Apr 27 '10 at 2:18
  • 1
    Not an answer that meets your exact spec, but consider mounting the remote ftp servers using curlftpfs on the central server so they appear to be local filesystems and then use rsync to synchronise. Requires no changes on satellite servers and you end up with rsync capabilities. Apr 27 '10 at 7:48

I think you want to look into mirror mode of lftp.

Rsync doesn't work over ftp (that would be the normal 'default' admin choice).

I strongly recommend you migrate to shared hosting that supports ssh terminal access which you can then use rsync over. By rolling out your own solution, you are only digging your hole deeper into this crappy (Your words) setup. I imagine the same amount of effort and time (if not less) to migrate to a better shared hosting provider.


Sounds like a job for weex?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.