Server Fault is a question and answer site for system and network administrators. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I need to deploy an automated process (via 1 min cron script) that looks for tar files in a specific directory. If a tar file is found, it is untarred to the appropriate location and then the tar file is deleted.

The tar files are automatically copied to this server over SSH from another server. In some cases, the tar files are extremely large, with lots of files.

The problem that I am expecting to run into: If it takes > 1 minute for the tar file to be copied to the server, and the cron script runs once every minute, it's going to see the .tar.gz file and try to do untar it, even though the tar file is still in the process of being written to.

Is there any way (via bash commands) to test if a file is currently being written to, or if it's only a partial file, etc?

One alternative I was thinking of was to have the file be copied as a different file extension (like .tar.gz.part) and then renamed to .tar.gz after the transfer is complete. But I figured I'd try to figure out if there is simply a way to determine if the file is whole at the command line first... Any clues?

share|improve this question
How exactly is the file being transferred? For example, rsync uses a temporary filename during the transfer (by default), and only after the file is completely transferred, renames it to the actual filename. – Piskvor Mar 27 '14 at 15:07
up vote 4 down vote accepted

You are on the right track, renaming the file is an atomic operation, so performing the rename after upload is simple, elegant and not error prone. Another approach I can think of is to use lsof | grep filename.tar.gz to check if the file is being accessed by another process.

share|improve this answer
Do you mean 'not' error prone? – Jake Wilson Aug 8 '12 at 16:09
Yeah, thanks for correction! – Alex Aug 8 '12 at 16:10
Checking the process is a good way to do it but I think I'm going to go with the renaming route because it's so simple and fool proof. – Jake Wilson Aug 8 '12 at 16:27
(lsof filename.tar.gz is more efficient and more accurate than lsof | grep filename.tar.gz) – Rich Nov 26 '15 at 10:50

Your best bet is to use lsof to determine if a file has been opened by any process:

#  lsof -f -- /var/log/syslog
rsyslogd 1520 syslog    1w   REG  252,2    72692 16719 /var/log/syslog

You can't easily tell if it's in the process of being written to, but if it is being written to, it MUST be open.

Edit: let's solve the actual problem here rather than try to implement the proposed solution!

Use rsync to transfer the file:

○ → rsync -e ssh remote:big.tar.gz .

This way, the file won't be copied over top of the existing one but copied into a temporary file (.big.tar.gz.XXXXXX) until transfer is complete, then moved into place.

share|improve this answer

The best way to do this is to use incron ("inotify cron system"). It allows you to set an inotify watch on a directory which will then notify you of file operations. In this case, you should watch the dir for a close_write. That'll allow you to then run your command once the file was closed after a write.

share|improve this answer

It seems like lsof can detect what mode a file is open under:

lsof -f -- a_file
cat     52391 bob    1w   REG    1,2       15 19545007 a_file

See where it says 1w? That means that the file descriptor number is 1 and the mode is w, or write.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.