Server Fault is a question and answer site for system and network administrators. It only takes a minute to sign up.
Sign up to join this community
I have several processes stuck on uninterruptible sleep statuses, all seemingly stemming from auplink /var/lib/docker/aufs/mnt. It's something docker related and it's waiting on an I/O that will never complete -- I get that, but how do I determine the exact cause? How can I know what I/O it is waiting on? Also, is there really no way of making these stuck processes go away without a hard reboot?
You can see stack of the process:
cat /proc/<process pid>/stack
which will give you information on what it was doing when it ended up in D-state.
echo w > /proc/sysrq-trigger; dmesg
will tell kernel to report all stack traces for D-state processes in dmesg buffer.
Processes in D-state cannot be killed. There are situations where process stays in D-state for long time but occasionally finishes I/O and is interruptible for short period of time and then goes back to the same I/O activity and ends up in D-state again. Then with
while (true); do kill -9 PID; done
there is a little chance of delivering KILL signal while process is interruptible.
strace -p <pid>Also,lsof |grep <pid>will show open file handles... also, you could try gdb.lsofgets stuck (you can see from above screenshot that it also goes into uninterruptible sleep status`. )