I have a script which will upload files from a specific directory (with rsync) to the server, but in the mean time another script will pull files from the server (also with rsync and with the --delete flag).

This means, that when I add a lot of files to it and rsync is in the process of uploading files and the download script is also being run, some files (which are not yet uploaded to the server) are deleted.

Is there any way I can prevent this? I want both my upload script and download script work simultaneously.

  • What exactly are you trying to achieve here? – mgorven May 21 '12 at 18:42

It sounds like you're trying to perform a two-way sync, in which case it's probably better to use a tool specifically designed for this, like unison.

  • Excellent, this is what I needed! – Devator May 21 '12 at 19:16

Make your upload work atomic. For example you can do rsync upload to temporary directory, and replace working directory with mv on finish. Another thing to try is to use version control to manage your files. I'd recommend git.

  • I'm actually developing something like Dropbox (but for personal use and learning purposes) so the files should be up- and downloaded simultaneously. – Devator May 21 '12 at 16:04
  • Than you should learn distributed software mechanisms: atomicity, locking, transaction isolation – DukeLion May 21 '12 at 18:51

If I have correctly understood the problem, you could simply exlude the partial files from being downloaded. This will also make sure they are not deleted. Check the --temp-dir switch in the rsync man page on how to specify a temporary directory which will be used while the files are being uploaded.

If you want to be able to up- and download a file at the same time, then you will need to create your own version of rsync which takes into account that requirement.

  • No, what I actually mean is that for example: you place 20 files in the folder (none of these 20 exist at the server), 5 of them are currently being uploaded and the download script also starts right now. The 15 files (which don't exist yet at the server) will be deleted in your folder, so only the first 5 which are (being) uploaded will be fine. – Devator May 21 '12 at 18:35
  • This is most confusing. Why would anybody want to do anything like that? – Oliver May 21 '12 at 18:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.