Since a recent linux kernel upgrade (3.2.78 on Debian Wheezy) on both my NFS clients and NFS server, I'm seeing really high increase (10 to 20 folds) in the number of nfs_lookup requests from the clients to the server. This is making a huge performance degradation to the application running on my NFS clients (apache+php-fpm).

What can be the cause of such a high increase of nfs_lookup?

Output of mountstats --nfs on one of the clients after about 1h of uptime is as follows:

Stats for ruchbah.nfs://membres mounted on /srv/membres:
  NFS mount options: rw,vers=4,rsize=1048576,wsize=1048576,namlen=255,acregmin=3,acregmax=60,acdirmin=30,acdirmax=60,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=,minorversion=0,local_lock=none
  NFS server capabilities: caps=0x7fff,wtmult=512,dtsize=32768,bsize=0,namlen=255
  NFSv4 capability flags: bm0=0xfdffbfff,bm1=0xf9be3e,acl=0x3
  NFS security flavor: 1  pseudoflavor: 0

Cache events:
  data cache invalidated 1231 times
  attribute cache invalidated 23693734 times
  inodes synced 1790 times

VFS calls:
  VFS requested 46296214 inode revalidations
  VFS requested 35900771 dentry revalidations

  VFS called nfs_readdir() 3678 times
  VFS called nfs_lookup() 35200320 times
  VFS called nfs_permission() 11201 times
  VFS called nfs_file_open() 22547 times
  VFS called nfs_file_flush() 1400 times
  VFS called nfs_lock() 7611 times
  VFS called nfs_fsync() 65321 times
  VFS called nfs_file_release() 3731 times

VM calls:
  VFS called nfs_readpage() 856 times
  VFS called nfs_readpages() 6945 times
  VFS called nfs_writepage() 10789 times
  VFS called nfs_writepages() 945872 times

Generic NFS counters:
  File size changing operations:
    truncating SETATTRs: 65272  extending WRITEs: 0
  291 silly renames
  short reads: 11201  short writes: 17
  NFSERR_DELAYs from server: 0

NFS byte counts:
  applications read 162084951 bytes via read(2)
  applications wrote 19636930 bytes via write(2)
  applications read 0 bytes via O_DIRECT read(2)
  applications wrote 0 bytes via O_DIRECT write(2)
  client read 152501261 bytes via NFS READ
  client wrote 20783074 bytes via NFS WRITE

The number of VFS calls (nfs_lookup in particular) is very high. The cache attribute invalidation is also abnormally high. The clients mostly uses the NFS mount to read data.

  • did you was using nfsv4 with the older version as well? – kofemann Jul 23 '16 at 20:12
  • Server and clients are NFS v4 only. – Tonin Jul 23 '16 at 20:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.