1

I am using a local website which downloads files to my server. Using an Inotify daemon I'm listening for file changes in a specific folder (the one I am downloading to) and executing a script which moves those files into another folder using rsync. However my problem here is that rsync moves the files before they are completely downloaded so they are at 0KB. I used mv before, and mv somehow managed to wait until the download is completed before moving, but I needed to use rsync because mv won't write to existing, non-empty folders.

Any ideas?

Here is my script which executes on inotify create:

#!/bin/sh

cd the_folder_which_is_watched_by_inotify

rsync -vh * /where_the_files_are_moved_to

echo "Files have been moved"
5
  • Not sure what you mean by this mv won't write to existing, non-empty folders.
    – Zoredache
    May 25, 2016 at 17:15
  • @Zoredache MV returns mv: cannot move './dir' to './dir2': Directory not empty when writing to folders containing files
    – PoTTii
    May 25, 2016 at 18:14
  • Why are you trying to move the folder, instead of moving the file?
    – Zoredache
    May 25, 2016 at 18:17
  • Because my download tool bulk downloads files and creates multiples folders with those files in it.
    – PoTTii
    May 25, 2016 at 18:20
  • Ok, but why would your inotify command try to move the folders instead of the individual files? Can you setup inotify to ignore folders and only work with the individual files?
    – Zoredache
    May 25, 2016 at 19:24

2 Answers 2

1

The reason mv would probably work, is because the file would actually be the same file. So the upload process would be writing to a given inode, and the mv command simply updates the source and destinations so that the inode is in a different location. But any programs with an open handle to the file would be able to continue working with that inode, so you can move the link around to wherever you want.

What you might be able to do is employ the rsync --link-dest option. Getting the paths right is tricky since --link-dest needs the relative path to the source from the perspective of the destination. But it should result in rsync making hardlinks to the file. That is both the source and destination should link to the exact same inode.

1

You should setup your inotify daemon to launch rsync only on CLOSE_WRITE and MOVE_FROM/MOVE_TO combo events.

If you are listening for CREATE and/or MODIFY events, you will end with rsync called when the file is being written to, which will led to a corrupt/empty copy.

2
  • Great, I am going to try that out later, thank you. What do you mean by MOVE_FROM/MOVE_TO combo events?
    – PoTTii
    May 26, 2016 at 7:59
  • When moving/renaming a file, the inotify subsystem generate two events: a MOVE_FROM (with the original file name/path) and a MOVE_TO (with the new name/path). These two events are emitted as two separate events, but they really form a single, combined event that has to be reconstructed using the coockie value embedded in these events. For more information, man inotify is your friend ;)
    – shodanshok
    May 26, 2016 at 9:15

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .