I've got a script that continuously monitors an FTP dropbox directory and processes new files when they come in. The script runs
lsof on each file before it tries to process it to determine if the file is still being transferred (some files are large and take a while to transfer) to avoid processing an incomplete file.
Everything has been working fine, except with this one specific FTP client. For some reason, when this client uploads a file to the FTP server, running
lsof on the file returns nothing. I tested this by running
watch -n0.1 'lsof /path/to/file.txt' and then uploading 'file.txt' to the server using two different FTP clients. When uploading with Cyberduck, lsof returns data about the open file, when using the other FTP client, lsof returns nothing. I don't understand how that could be ( and FWIW, I get the same results with the
fuser command). I'm also running
watch -n0.1 'ls -l /path/to/file' and can see the size of the file changing as the clients are transferring, so I know the clients are writing to the file.
I know there are workarounds to solve my problem, but I'm more interested in exactly how this FTP client can be writing to a file without that file being 'open' as far as the system is concerned. And ideas?
I'm running vsftp on Ubuntu Server 12.04 LTS
It turns out that when I tested this earlier using Cyberduck, I was connecting to the FTP server using SFTP, not FTP. When I upload a file with Cyberduck using FTP, I run into the same problem--the file uploads, but nothing shows up in
lsof. I also used the standard FTP client in Debian to upload a file using PUT and APPEND--again, same thing.
It looks like all FTP uploads (regardless of client) write to their respective file without "opening" the file as far as the system is concerned. I tested this scenario on another FTP server, and everything works as expected--when the files are being uploaded, they show as open in lsof. So, there is something wrong with the specific configuration of my FTP server. I suspect it has something to do with the version of lsof and/or vsftpd. They are older versions, 4.81 and 2.3.2 respectively.