Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

i've got a problem removing mounts created with mount -o bind from a locally mounted NFS folder. Assume the following mount structure:

NFS mounted directory:

$ mount -o rw,soft,tcp,intr,timeo=10,retrans=2,retry=1 \
 10.20.0.1:/srv/source /srv/nfs-source

Bound directory:

$ mount -o bind /srv/nfs-source/sub1 /srv/bind-target/sub1

Which results in this mount map

$ mount
/dev/sda1 on / type ext3 (rw,errors=remount-ro)
# ...
10.20.0.1:/srv/source on /srv/nfs-source type nfs (rw,soft,tcp,intr,timeo=10,retrans=2,retry=1,addr=10.20.0.100)
/srv/nfs-source/sub1 on /srv/bind-target/sub1 type none (rw,bind)

If the server (10.20.0.1) goes down (eg ifdown eth0), the handles become stale, which is expected.

I can now un-mount the NFS mount with force

$ umount -f /srv/nfs-source

This takes some seconds, but works without any problems. However, i cannot un-mount the bound directory in /srv/bind-target/sub1. The forced umount results in:

$ umount -f /srv/bind-target/sub1
umount2: Stale NFS file handle
umount: /srv/bind-target/sub1: Stale NFS file handle
umount2: Stale NFS file handle

Here is a trace http://pastebin.com/ipvvrVmB

I've tried umounting the sub-directories beforehand, find any processes accessing anything within the NFS or bind mounts (there are none).

lsof also complains:

$ lsof -n
lsof: WARNING: can't stat() nfs file system /srv/nfs-source
      Output information may be incomplete.
lsof: WARNING: can't stat() nfs file system /srv/bind-target/sub1 (deleted)
      Output information may be incomplete.
lsof: WARNING: can't stat() nfs file system /srv/bind-target/
      Output information may be incomplete.

I've tried with recent stable Linux kernels 3.2.17, 3.2.19 and 3.3.8 (cannot use 3.4.x, cause need the grsecurity patch, which is not, yet, supported - grsecurity is not patched in in the tests above!).

My nfs-utils are version 1.2.2 (debian stable).

Does anybody have an idea how i can either:

  • force the un-mount some other way? (any dirty trick is welcome, data loss or damage neglible at this point)
  • use something else instead of mount -o bind? (cannot use soft links, cause mounted directories will be used in chroot; bindfs via FUSE is far to slow to be an option)

Thanks, Paul

Update 1

  • With 2.6.32.59 the umount of the (stale) sub-mounts work just fine. It seems to be a kernel regression bug.
  • The above tests where with NFSv3. Additional tests with NFSv4 showed no change.

Update 2

  • We have tested now multiple 2.6 and 3.x kernels and are now sure, that this was introduced in 3.0.x. We will fille a bug report, hopefully they figure it out.
share|improve this question
1  
Try umount -l –  Dmitri Chubarov Jun 7 '12 at 13:24
    
@Dimitri: Sorry, did not mention this. I've tried, does not work. Neither does umount -lf. –  Paul Eisner Jun 7 '12 at 15:26
    
What happens if you unmount the /srv/bind-target/sub1 first and then is /srv/nfs-source? –  quanta Jun 8 '12 at 4:58
    
@quanta: Same thing. Also the trace looks exactly the same. –  Paul Eisner Jun 8 '12 at 9:22
    
Is the question on how to handle the current situation without rebooting, or on how to make the system resilient to that problem in the future? –  rackandboneman Jun 8 '12 at 11:41
show 1 more comment

3 Answers

You could simply mount the remote filesystem on /srv/bind-target/sub1.

If you're expecting this level of unavailability you should also specify the sync (although maybe defailt) option to NFS to lower the chance of having unwritten changes on your client

share|improve this answer
add comment

Something that worked for me in my particular setup, to get cleints working, was this:

I had an autofs tree with a mounted nfs fs on /fs/doom and another mounted on /fs/doom/localvol5. After server reboot it was possible to access /fs/doom and /fs/doom/localvol5/sub, but /fs/doom/localvol5 itself gave ESTALE on everything, including umount -f, -l, -fl.

What I did to get the client running without reboot was to move the whole /fs/doom hierarchy to another tree:

    mkdir /dev/shm/garbage-mount
    mount --move /fs/doom /dev/shm/garbage-mount

That moved the whole tree, and apparently works only because /fs/doom was accessible and a mountpoint. I sitll cannot unmount any of these filesystems, but I was able to restart autofs and get a fresh and working tree.

This should work with any autofs tree that has malfunctioning nfs subdirs.

Hope that helps.

share|improve this answer
add comment
$ umount -l

Worked for me, before none of them were working:

$ mount -f ...
share|improve this answer
2  
Posting blind commands is generally a poor way to answer questions - it encourages voodoo system administration. Please update your post to clarify what the commands do and why they're useful so the OP can actually learn something... –  voretaq7 Jun 17 '13 at 19:08
    
He also mention over a year ago that he's tried this. –  Aaron Copley Jun 17 '13 at 19:28
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.