You may want to use some kind of locks.
If you control writers (at least the way they are started) consider using
The simplest way is to use command line tool
- to run writer (or many writers) with
flock --shared <file> <writing_command> It's important to use
--shared here to allow several independent processes to acquire lock.
- to run your backup script with
flock --exclusive <file> <backup_script>
This will prevent your backup script from starting when shared lock is held by other processes. At the same time shared lock cannot be acquired when exclusive one is being held. That will delay the start of the writer until that lock is released
Command line solution has some limitations:
It will only work if at least one of the following is true:
- process (either writer or backup script) does not daemonize
- or it does not close it's file handles
Otherwise, when process looses lock by closing handle and daemonizes it's parent (the
flock process) also dies finally releasing that lock.
This is not very useful with long running processes. As processes are not aware of locking they cannot release that lock while running. So lock is continuously being held even if process does not write into file
Another ultimate, more complicated, but way more powerful solution is to implement all of the above inside file-writing code. This will let you acquire lock only for the required period, releasing it right after writing transaction is finished. Take a look into language-specific manuals about flock.
Just few thing you may want to know:
- locks are attached to file handles, so closing file will result in releasing lock
- locks are inherited, so all children of process are holding the same lock
- Inheritance of locks is a result of inheritance of handles. That also means that releasing lock on a handle in a child will - - these locks are voluntary and do not prevent direct access to file without acquiring a lock, so this will work only if every involved process acquires appropriate lock release lock on parent
You may also want to take a look at mandatory locking with
fcntl but it is known to be unreliable in current linux kernels