We've got a Centos 7 system doing some nfs serving that is using way more disk space than it should, that is it is 87% full, but du only accounts for about 20%. My assumption is that something is writing to a deleted file, probably over nfs as lsof isn't showing anything significant locally (actually, it's not showing anything significant on the nfs clients either). Rebooting the server would give 30ish clients a bunch of stale file handles and be very disruptive. Is there any other way to find the lost space?
1 Answer
First check open file which @Michael Hampton already suggested, to clean the stale files in normal case you do no need to reboot server, for that find which applications using those files and restart those applications only.
Can you share more details with logs like apps details, disk usage output mount
command output etc.
-
1As I mentioned, I already tried lsof, but it doesn't seem to show any files opened over nfs; this server is a log server, and I suspect the primary application, which doesn't use syslog, is the culprit, so I'll have to restart it on all the client servers. As they are in use 24x7, it'll be disruptive and it would be nice to know for sure, though I suspect they're all culprits and will all have to be restarted anyhow.– abatieAug 28, 2019 at 4:31
lsof
on the NFS server.