8

There is a lot of confusing information out there on pNFS, and I have some very simple questions:

  • How can I check if it is running/enabled/active on my system?
  • Is pNFS widely available today or still experimental? In particular what is its status on Ubuntu 18.04? Does nfs-kernel-server use pNFS?
  • What is the difference/relationship between NFSv4.1 and pNFS?
  • Is pNFS an optional part of NFSv4.1 or is it always active when using NFSv4.1?
  • Is pNFS also a part of NFSv4.2 or just 4.1?
  • pNFS is so called because it enables parallel access from client to server. I assume this means that without pNFS a client only has a single NFS process running that connects to the server by a single TCP connection. Is this correct?
6
  • 2
    I don’t think anyone uses pNFS. What’s your use case?
    – ewwhite
    Sep 26, 2019 at 12:02
  • Use case is a storage server with about 20 clients. Some clients are compute servers with many parallel processes accessing the storage. I thought pNFS could help with establishing multiple parallel connections to the server, as we already have multiple parallel Ethernet connections to the storage server by bonding. The performance is mostly sufficient, except for some times. I'm simply exploring options and trying to understand possible directions for scaling later. We also have different bottlenecks at the moment (server disk io), but again, I'd like to understand the NFS space a bit better.
    – isarandi
    Sep 26, 2019 at 12:07
  • And your comment is exactly what I am looking for: the status in practice right now. It's easy to find powerpoint slides from 5 years ago and RedHat docs talking about it, but it's very hard to find the practical status.
    – isarandi
    Sep 26, 2019 at 12:09
  • 1
    Try to identify your bottleneck. You can increase NFS daemons, add bandwidth, etc.
    – ewwhite
    Sep 26, 2019 at 12:31
  • 2
    @isarandi You don't need pNFS for parallel processing of your multiple concurrent NFS requests. Even NFSv3 will work this way just fine. Sep 26, 2019 at 12:46

1 Answer 1

6

pNFS is an optional part of NFSv4.1 spec. IOW, you need nfs server and client to talk nfsv4.1+ to make use of it. As this is an option functionality, your client and server should support it. Linux kernel support pNFS since version 3.9 and with every new release it's get more mature.

In general, pNFS allows to spread request for a single file to multiple so called data-servers. For example, an application issues 16MB read, but the NFS server sends 2MB reads to 8 different servers that have required blocks. There are several server implementations that support pnfs:

  • Linux server; supports only block layout
  • Gangesha nfs based implementations
  • NetApp 8.1+
  • HammerSpace
  • dCache (the project where I am involved)
  • FreeBSD

I can't talk about other systems, but we are moving terabytes of data per day on systems with hundreds of pNFS data-servers exposed as a single NFS server.

Check your NFS server for pNFS capability. On the client side, ensure that the server is mounted with nfsv4.1 or above. If the server is pNFS capable, then you should see the appropriate layout driver loaded as a kernel module:

$ lsmod | grep layout
nfs_layout_nfsv41_files    36864  0
nfs_layout_flexfiles    53248  0
nfsv4                 708608  11 nfs_layout_flexfiles,nfs_layout_nfsv41_files
nfs                   323584  3 nfsv4,nfs_layout_flexfiles,nfs_layout_nfsv41_files
sunrpc                454656  22 nfsv4,auth_rpcgss,nfs_layout_flexfiles,lockd,nfs_layout_nfsv41_files,rpcsec_gss_krb5,nfs
$

as well as the corresponding kernel message:

$ dmesg  | grep Layout
[41827.049921] nfs4flexfilelayout_init: NFSv4 Flexfile Layout Driver Registering...
$

The Layout type specifies how clients talk to various data server. Check the Storage Protocols section in RFC 5661.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.