35

... to compensate for broken DNS servers that are outside our control.

Our problem: We deploy embedded devices that collect sensor data at various, mostly IPv4-only sites. Some sites have poorly maintained networks, e.g. misconfigured or otherwise broken DNS caches and/or firewalls that either ignore AAAA queries altogether, or respond to them with broken replies (e.g. wrong source IP!). As an external supplier to the facilities department, we have next to no influence on the (sometimes reluctant) IT departments. The chances of them fixing their DNS servers/firewalls any time soon are minuscule.

The effect on our device is that with each gethostbyname(), the processes have to wait until the AAAA queries time out, at which point some processes have already timed out their connection attempts altogether.

I am looking for solutions that are ...

  • system-wide. I cannot reconfigure dozens of application individually
  • non-permanent and configurable. We need to (re-)enable IPv6 where/when it gets fixed/rolled out. Reboot is OK.
  • If a solution requires a core library like glibc to be replaced, the replacement library package should be available from a known-to-be-well-maintained repository (e.g. Debian Testing, Ubuntu universe, EPEL). Self-building is not an option for so many reasons that I don't even know where to begin with, so I just don't list them at all...

The most obvious solution would be to configure the resolver library e.g. via /etc/{resolv,nsswitch,gai}.conf to not query AAAA records. A resolv.conf option no-inet6 as suggested here would be exactly what i am looking for. Unfortunately it is not implemented, at least not on our systems (libc6-2.13-38+deb7u4 on Debian 7; libc6-2.19-0ubuntu6.3 on Ubuntu 14.04)

So how then? One finds the following methods suggested on SF and elsewhere, but non of them work:

  • Disabling IPv6 altogether, e.g. by blacklisting the ipv6 LKM in /etc/modprobe.d/, or sysctl -w net.ipv6.conf.all.disable_ipv6=1. (Out of curiosity: Why is the resolver asking for AAAA where IPv6 is disabled?)
  • Removing options inet6 from /etc/resolv.conf. It wasn't there in the first place, inet6 is simply enabled by default these days.
  • Setting options single-request in /etc/resolv.conf. This only ensures that the A and the AAAA queries are done sequentially rather than in parallel
  • Changing precedence in /etc/gai.conf. That does not affect the DNS queries, only how multiple replies are processed.
  • Using external resolvers (or running a local resolver daemon that circumvents the broken DNS servers) would help, but is usually disallowed by the company's firewall policies. And it can make internal resources inaccessible.

Alternative ugly ideas:

  • Run a DNS cache on localhost. Configure it to forward all non-AAAA queries, but to respond to AAAA queries with either NOERROR or NXDOMAIN (depending on the result of the corresponding A-query). I am not aware of a DNS cache able to do this though.
  • Use some clever iptables u32 match, or Ondrej Caletka's iptables DNS module to match AAAA queries, in order to either icmp-reject them (how would the resolver lib react to that?), or to redirect them to a local DNS server that responds to everything with an empty NOERROR.

Note that there are similar, related questions on SE. My question differs insofar as it elaborates the actual problem i am trying to solve, as it lists explicit requirements, as it blacklists some often-suggested non-working solutions, and as it is not specific to a single application. Following this discussion, I posted my question.

  • 13
    PS: Contrary to popular belief here on SF, there are some good reasons to disable IPv6/AAAA on a machine in a IPv4-only network, even where DNS works: Reduce broadcast load; Reduce load on DNS resolvers by almost 50%; Reduce connection start-up times (significantly where DNS caches are laggy); Follow best practices to disable non-functional features to enhance security and stability. Admittedly, if i forget to re-enable IPv6 once it becomes available, then my system becomes IPv4 legacy ballast that impedes IPv6 rollout. One should be allowed to weigh the listed pros against this con. – Nils Toedtmann Oct 1 '14 at 9:33
  • Any reason why you don't run a full resolver on localhost? That way you eliminate the dependency on other people's (seemingly) unreliable DNS resolvers altogether. – Sander Steffann Oct 1 '14 at 9:47
  • @SanderSteffann Company firewall policies usually disallow that. But elsewhere that is an option. I'll add it to my question later. – Nils Toedtmann Oct 1 '14 at 9:56
  • 3
    @joeqwerty We are not making any assumptions about whether or not IPv6 is supported on site. We do make the assumption though that DNS servers are standard-compliant. Also, some IT departments unfortunately do lack the skills to configure their infrastructure properly. Sorry for being blunt about that. – Nils Toedtmann Oct 1 '14 at 12:50
  • 4
    I understand what you're saying. You need to make your box work on their network not the other way around, and that is what you're here asking about. I just get a little miffed when we in the IT field blame our customers and don't respect them. They are our bread and butter. For good or bad we need to respect that and respect them. Our customers are not a hindrance to our business, they are the reason for our business. – joeqwerty Oct 1 '14 at 13:01
9

Stop using gethostbyname(). You should be using getaddrinfo() instead, and should have been for years now. The man page even warns you of this.

The gethostbyname*(), gethostbyaddr*(), herror(), and hstrerror() functions are obsolete. Applications should use getaddrinfo(3), getnameinfo(3), and gai_strerror(3) instead.

Here is a quick sample program in C which demonstrates looking up only A records for a name, and a Wireshark capture showing that only A record lookups went over the network.

In particular, you need to set ai_family to AF_INET if you only want A record lookups done. This sample program only prints the returned IP addresses. See the getaddrinfo() man page for a more complete example of how to make outgoing connections.

In the Wireshark capture, 172.25.50.3 is the local DNS resolver; the capture was taken there, so you also see its outgoing queries and responses. Note that only an A record was requested. No AAAA lookup was ever done.

#include <sys/types.h>
#include <sys/socket.h>
#include <string.h>
#include <stdlib.h>
#include <netdb.h>
#include <stdio.h>

int main(void) {
    struct addrinfo hints;
    struct addrinfo *result, *rp;
    int s;
    char host[256];

    memset(&hints, 0, sizeof(struct addrinfo));
    hints.ai_family = AF_INET;
    hints.ai_socktype = SOCK_STREAM;
    hints.ai_protocol = 0;

    s = getaddrinfo("www.facebook.com", NULL, &hints, &result);
    if (s != 0) {
        fprintf(stderr, "getaddrinfo: %s\n", gai_strerror(s));
        exit(EXIT_FAILURE);
    }

    for (rp = result; rp != NULL; rp = rp->ai_next) {
        getnameinfo(rp->ai_addr, rp->ai_addrlen, host, sizeof(host), NULL, 0, NI_NUMERICHOST);
        printf("%s\n", host);
    }
    freeaddrinfo(result);
}
  • Interesting! I will investigate which applications trigger AAAA requests. If it's only ours, then i'll relay your suggestion to our devs. But i have a strong hunch that a lot of Debian/Ubuntu-packaged software is suffering from this, and we ain't gonna patch those. – Nils Toedtmann Jul 27 '15 at 9:13
  • Your application is probably the most important one. Even if you can't fix everything else, it may improve the situation. – Michael Hampton Jul 27 '15 at 13:24
4

When in doubt, head over to the source code! So, let's see... gethostbyname() looks interesting; that describes exactly what we're seeing: try IPv6 first, then fall back to IPv4 if you don't get an answer you like. What's this RES_USE_INET6 flag? Tracing it back, it's coming from res_setoptions(). This is where resolv.conf is read in.

And.... that's me out of ideas. I'm completely unclear how it is that RES_USE_INET6 is being set if not in resolv.conf.

  • RES_USE_INET6 can be set via options inet6 in resolv.conf. I guess my problem is that it cannot be unset once it was set at compile time, which all major distros seem to do these days (correct?). Therefore the feature request for options no_inet6 that i mentioned above. – Nils Toedtmann Oct 3 '14 at 7:03
  • 1
    Unfortunately, as you can see from the code, there doesn't appear to be a no_inet6 option in res_setoptions(). However, as you can see from (no-)ip6-dotint, it's an easy change to add. To test the theory that it's being set by default by your distro, I'd grab the package source files and compile it once "virgin" (to confirm that the package replicates the behavior) and then add: { STRnLEN ("no-inet6"), 1, ~RES_USE_INET6 }, to the options[] array and see if the problem goes away when you set that option in resolv.conf. – BMDan Oct 6 '14 at 21:24
  • 1
    Lastly: for what it's worth, I'd solve this by running a DNS cache on localhost (as you reference, above). It'd be a heckuva lot easier to maintain your own, hacked DNS proxy/cache than it would be to maintain a hacked version of a core system library. – BMDan Oct 6 '14 at 21:26
3

You could use BIND as a local resolver, it has an option to filter AAAA:

https://kb.isc.org/article/AA-00576/0/Filter-AAAA-option-in-BIND-9-.html

  • 1
    That's pretty heavyweight for an embedded device. – Michael Hampton Jul 25 '15 at 21:25
  • Thank you, i wasn't aware of Bind's AAAA filter. As Michael mentions, it's probably not a solution for us due to Bind's large footprint. But for those who want to filter out AAAA responses in other scenarios it might be a viable way. Ubuntu actually builds bind with "--enable-filter-aaaa", at least on 14.04. Not sure about Debian. - See also ipamworldwide.blogspot.co.uk/2011/09/… – Nils Toedtmann Jul 26 '15 at 20:20
  • 1
    I am on 14.04 and it does not seem that this filtering option is available. – Zitrax Apr 6 '17 at 15:04
0

Did you try to setup PDNS-recursor, set it in your /etc/resolv.conf and deny "AAAA" lookups in it? Using something like query-local-address6=

  • 1
    query-local-address6= does something different (which IPv6 address to send queries from - note that even with IPv6 disabled, AAAA requests will still be resolved via IPv4). Also I cannot identify any other setting that would filter AAAA queries (doc.powerdns.com/html/built-in-recursor.html). Without that information, your answer is not very helpful :( – Nils Toedtmann Oct 2 '14 at 10:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.