2

Need command in linux to determine, if the file is getting written into it.

We have used a script to backup of data and it is working fine. Now I need a smart way to copy the backedup file (say file.csv to get copied to another location). So I need a command that makes sure if the file (file.csv) is completed its writing, upon writing only, i need to copy this to another location.

Thanks for your help in advance. If it is confusing I can elaborate it.

3 Answers 3

2

What you want is Incron:

http://inotify.aiken.cz/?section=incron&page=about&lang=en

Incron can listen for specific filesystem events and perform actions on matching events.

2

You can use the status of lsof or fuser to determine if the file is open.

Something like:

lsof /path/to/csv.txt || backup_script.sh or
fuser /path/to/csv.txt || backup_script.sh in a loop would be the equivalent of: Run backup script only if this file isn't open.

2

You may want to use some kind of locks.

If you control writers (at least the way they are started) consider using flock.

The simplest way is to use command line tool flock:

  • to run writer (or many writers) with flock --shared <file> <writing_command> It's important to use --shared here to allow several independent processes to acquire lock.
  • to run your backup script with flock --exclusive <file> <backup_script>

This will prevent your backup script from starting when shared lock is held by other processes. At the same time shared lock cannot be acquired when exclusive one is being held. That will delay the start of the writer until that lock is released

Command line solution has some limitations:

  1. It will only work if at least one of the following is true:

    • process (either writer or backup script) does not daemonize
    • or it does not close it's file handles

    Otherwise, when process looses lock by closing handle and daemonizes it's parent (the flock process) also dies finally releasing that lock.

  2. This is not very useful with long running processes. As processes are not aware of locking they cannot release that lock while running. So lock is continuously being held even if process does not write into file

Another ultimate, more complicated, but way more powerful solution is to implement all of the above inside file-writing code. This will let you acquire lock only for the required period, releasing it right after writing transaction is finished. Take a look into language-specific manuals about flock.

Just few thing you may want to know:

  • locks are attached to file handles, so closing file will result in releasing lock
  • locks are inherited, so all children of process are holding the same lock
  • Inheritance of locks is a result of inheritance of handles. That also means that releasing lock on a handle in a child will - - these locks are voluntary and do not prevent direct access to file without acquiring a lock, so this will work only if every involved process acquires appropriate lock release lock on parent

You may also want to take a look at mandatory locking with fcntl but it is known to be unreliable in current linux kernels

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .