2

I have a problem with nfs client not being able to resolve a resolvable DNS name.

[root@testserver-2 ~]# host nfs-server-host-name
nfs-server-host-name has address 10.37.4.131

[root@testserver-2 ~]# nslookup nfs-server-host-name
Server:         127.0.0.1
Address:        127.0.0.1#53

Name:   nfs-server-host-name
Address: 10.37.4.126

[root@testserver-2 ~]# showmount -e nfs-server-host-name
clnt_create: RPC: Unknown host

[root@testserver-2 ~]# ss -lnp |grep rpc
LISTEN     0      128                       *:111                      *:*      users:(("rpcbind",7627,8)

[root@testserver-2 ~]# mount -t nfs -o defaults,auto,proto=tcp nfs-server-host-name:/ifs/exports/EXPORT /mnt/export
mount.nfs: Failed to resolve server nfs-server-host-name: Name or service not known

Local resolving is set up via dnsmasq:

[root@testserver-2 ~]# cat /etc/resolv.conf
options rotate timeout:2 attempts:4
nameserver 127.0.0.1
nameserver 10.1.1.1
nameserver 8.8.8.8

[root@testserver-2 ~]# cat /etc/dnsmasq.conf 
resolv-file=/etc/resolv.conf
server=/nfs-server-host-name/10.37.4.1 #IP address of Isilon smart connect resolver
listen-address=127.0.0.1

nsswitch configuration (default Centos 6):

[root@testserver-2 ~]# grep hosts /etc/nsswitch.conf
hosts:      files dns

Please note that nfs-server-host-name resolving is handled by an Isilon smart connect resolver, which is configured as a resolver for nfs-server-host-name locally via dnsmasq and works (as seen in the above example). Replacing nfs-server-host-name with an IP address is not an option as there are multiple NFS nodes which are balanced by the resolver therefore the IP is variable and cannot be hardcoded. It is therefore OK that the two results above from hosts and nslookup are different. This is intended and expected behaviour.

Please also note that nfs-server-host-name is a very precise example of what the hostname looks like. The actual hostname looks almost identically, it's not a FQDN. In fact there are no dots in the hostname at all. This is a feature I'm not in control of.

When mounted manually with one valid node IP address the export mounts fine. When one node's address is put to /etc/hosts, it mounts. When resolved via DNS means, it does not work for the nfs client but works for other net tools such as hosts, dig or ping.

This seems to be not-so-uncommon problem but all tips I found so far say "replace hostname with IP" which is something I cannot do.

What I missed?

  • 1
    What does nslookup return? – sysfiend May 11 '16 at 12:39
  • nslookup behaves as expected, responds with IP address of the host and 127.0.0.1 as the dns server queried (which is ok, it's the local dnsmasq). I'll add the output to the question, thanks – mikky May 11 '16 at 12:41
  • What does the hosts entry in your /etc/nsswitch.conf say? If it has only files, or something like files [NOTFOUND=return] dns, the libc resolver will never even try to use DNS. – Guntram Blohm May 11 '16 at 13:08
  • hosts: files dns, that's it. Will add to the question. – mikky May 11 '16 at 13:17
2

The showmount command has an RPC call which uses gethostbyname_r to attempt to get information for a hostname. It does not do much to interpret errors returned by it. Could you run a test to see what the error actually is? This code is modified from the actual glibc clnt_gen.c code seen here

Example C source:

#include <netdb.h>
#include <alloca.h>
#include <stdio.h>
#include <stdlib.h>
#include <errno.h>

int main(int argc, char **argv)
{
    struct hostent hostbuf, *h;
    size_t hstbuflen;
    char *hsttmpbuf;
    char *hostname;
    int herr;

    if (argc != 2)
    {
      exit(-1);
    }
    hostname = argv[1];

    hstbuflen = 1024;
    hsttmpbuf = alloca(hstbuflen);
    while (gethostbyname_r (hostname, &hostbuf, hsttmpbuf, hstbuflen, &h, &herr) != 0 || h == NULL)
    {
        if (herr != NETDB_INTERNAL || errno != ERANGE)
        {
            printf("gethostbyname_r error: %s\n", hstrerror(herr));
            exit(0);
        }
        else
        {
            hstbuflen *= 2;
            hsttmpbuf = alloca (hstbuflen);
        }
    }
}

Save this as ghbntest.c and compile with the command gcc -o ghbntest ghbntest.c. Run using ./ghbntest nfs-server-host-name. Output example:

$ ./ghbntest 12345.example.com
gethostbyname_r error: Unknown host
| improve this answer | |
  • thanks, it says gethostbyname_r error: Unknown host when there is no entry in /etc/hosts and nothing when the entry is there. One thing I may have been more clear about is that the example name nfs-server-host-name is actually a more precise example than one would think - there are actually no dots in the host name (and no search option in /etc/resolv.conf) – mikky May 12 '16 at 7:54
  • I had a hunch that it would have been related to the hostname you're providing not being a FQDN. I'll bet if you provide the hostname with a trailing period to the test program, you won't get an error. It seems in your other reply that that worked out to be the solution for your original NFS problem. – brent May 12 '16 at 14:00
  • The non-fqdn name was just a part of the problem, the other one being nfs client using incorrect nameserver to resolve the the name in the first place, as described in my answer. Both the problems however point to messed up dns resolving implemented in nfs client as it did ask the wrong server and filled in a search domain even though it was not supposed to. I have yet to figure out where did nfs client got the fqdn part - the only configuration where the entry was present was the host's hostname. Network is configured statically. – mikky May 13 '16 at 11:07
  • returncode 16, but no output, is what I'm getting. – erikbwork Jan 18 '18 at 18:09
2

I have yet to figure out what the real problem is but the solution is to remove rotate option from /etc/resolv.conf and put a dot after nfs-server-host-name in /etc/fstab to prevent domain search (which occurred even when there was no search option in /etc/resolv.conf), i.e.:

nfs-server-host-name.:/nfs/export/...
                    ^
                    ^---up here

It may have something to do with kernel build option CONFIG_NFS_USE_LEGACY_DNS, which is set to yes on my kernel.

The behaviour of nfs-client (and nfs-client only) was that it only queried the second nameserver in /etc/resolv.conf no matter how many there were as long as the rotate option was present. It did, however, work when there was no second nameserver at all. Beats me...

| improve this answer | |
  • I wonder if this also happens on more recent LTS kernels... – mikky May 13 '16 at 16:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.