0

I would like to transfer a directory of files to a remote destination. Easy.

However, at any given time, one or many of these files may be in-use and being updated by users.

This doesn't bother rsync - it will happily transfer the files ... but the resulting destination directory may have half-written or half-created files in it.

I thought of perhaps running 'lsof' or checking /proc filesystem to determine whether files were in use and running rsync if they are not, but that's just a race - maybe someone writes or opens right as I do the rsync.

Or, I could check 'lsof', then lock the directory with chmod and then run the rsync ... again, possible race condition and if someone does try to access, they will get errors / denied.

How might this be done ?

1 Answer 1

1

Are you using LVM?

You could do some kind of LVM snapshot. The snapshot will stop the files dead in their tracks whilst you dump an image of the snapshot and the users can carry on working, unaware that the system is being backed up.

Using an LVM snapshot would not prevent the file from being saved properly, though. What are the users doing? If it was a text file that was open in a text editor then a LVM snapshot would just preserve the file in the state that it is in on disk. But if your users are compiling binaries and stuff then you're gonna get files saved in unpredictable conditions.

2
  • Hmm... now that I think more about this, isn't this what the 'ci' and 'co' (check-in, check-out) commands are for ? That is, if the users could be instructed to co/ci the files, then we could rsync the dir and not care at all, since only the checked-out files would be in a temporary state, but the actual files would be fine ... isn't that what those old fashioned commands are for ?
    – user227963
    Jul 8, 2020 at 0:48
  • It's the first time I've heard of those commands. But yep. A good way to go about this is to try and get your users to work with good file handling techniques and, if they are not computer science graduates, try to explain to them why it is good to do things a certain way. I once sat and watched a friend on their own laptop uninstall a program in Windows that they weren't using any more. They actually went into the /Program Files (x86)/ directory, found the folder which contained the program and just deleted the folder! I had to explain to them that is not how to remove a software package! :-D
    – bitofagoob
    Jul 9, 2020 at 23:37

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .