Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I looked at a CDN and found i need to use SFTP to upload files to it. Using sftp isnt to difficult but how do i continuously upload files? my webapp current puts apps in /uploads/type/01/02/03/04/05/06/07/08.ext when the file id is 0x0102030405060708

How do i upload these files? do i launch sftp everytime a user uploads a file? is there some kind of deamon mode i can use to monitors when i add something to uploads and uses sftp on it? I was thinking about using cron but i dont want to check if thousands of files exist everytime. Also what happens when the connection has been terminated (lets say i restart my server for maintenance reasons). I dont know whats the easiest way to have everything work.


share|improve this question

rsync is a great tool for synchronizing lots of files with a remote site.

rsync runs over ssh so it will work if you have sftp access to your server. For an automated sync that runs from a cron job, you would setup ssh public key authentication so that the ssh connection does not prompt for a password.

share|improve this answer

Have you looked at using rdist via a crontab entry?

share|improve this answer
I never heard of that. Will it try to sync 100k+ files everytime cron is ran? If it does then its likely not suitable. – acidzombie24 Nov 14 '11 at 20:16
only the ones that are different between the server and the client – mdpc Nov 14 '11 at 20:20
well, thats what i mean. How does it know which are different? it has to check the modified time of 100k+ files right? that would take way to long – acidzombie24 Nov 14 '11 at 20:25

What platform does your web application run on?

If it's Linux (or *NIX in general) then rsync or rdist are good solutions.

If it's a Windows box, then I would recommend you have a loot at synchronization solutions that can observer and keep in sync entire folder trees, like Syncovery ( for example, which is my personal favorite. But there's a plethora of options out there, like GoodSync, BestSync, AllWaySync, just to name a few.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.