DNS hostname no longer resolves if a sub-hostname has been defined

Question

I have trouble understanding a specific effect in DNS name resolution for subdomain names that came to light when enabling lets encrypt.

The context of my question:

I have a domain (basjes.nl) and I have set all hostnames in that domain to resolve to the same IP address.

In the UI of my DNS provider (Transip) I see something like this:

@                      A      x.x.x.x
*                      CNAME  @

Now I can do this to get the specified IP address.

dig blurp.basjes.nl

Recently I deployed the ACME-DNS tool and setup the appropriate DNS entries (i.e. delegate auth.basjes.nl to my own ACME-DNS server) to allow Lets Encrypt to verify I own the specified hostname via DNS.

For this specific hostname I want to have a 'full name' HTTPS certificate (i.e. not the wild card) so I added something like this:

_acme-challenge.blurp  CNAME  something.auth.basjes.nl.

After adding this entry I find that doing dig blurp.basjes.nl no longer works. The strange thing is that I do not get an error (so no NXDOMAIN), I either get nothing or just

;; AUTHORITY SECTION:
basjes.nl.      300 IN  SOA ns0.transip.net. hostmaster.transip.nl. 2018041107 14400 1800 2419200 300

So at this moment I have something like this:

@                      A      x.x.x.x
*                      CNAME  @
_acme-challenge.blurp  CNAME  something.auth.basjes.nl.

I have found that if I add an explicit DNS record for this hostname then suddenly it DOES work again:

@                      A      x.x.x.x
*                      CNAME  @
blurp                  CNAME  @
_acme-challenge.blurp  CNAME  something.auth.basjes.nl.

The effects I would like to understand:

• Why DOESN'T it resolve anymore when asking a normal DNS server ?

  The '*' record is still present!

  dig blurp.basjes.nl @1.1.1.1

• Why DOES it resolve when asking directly the SOA DNS server of Transip?

  dig blurp.basjes.nl @ns0.transip.net

• Why DOES it resolve when adding the explicit record?

  This is what I currently use as a workaround to make it work for me.

P.S. I have contacted Transip support before posting this question. They didn't know what went wrong here and/or why this happens.

Answer 1

DNS is hierarchical. When you added the name _acme-challenge.blurp.basjes.nl, that meant that blurp.basjes.nl was also implicitly added. The bit here that may be surprising is that a name can exist without having any content. When you used dig to ask for blurp.basjes.nl, you got a perfectly correct and accurate answer: the name exists, but contains no records. This case, where a name exists only because there are other names below it in the DNS tree, is called an empty non-terminal. You can google that term if you want to know more.

That's the first part of your confusing behavior. The second part is this: a wildcard record is used to generate an answer if and only if the name asked for does not already exist. So, first you asked for blurp.basjes.nl and got an answer generated from the wildcard. Then you added _acme-challenge.blurp.basjes.nl. When you asked again after that, the name blurp.basjes.nl did exist (as described above), so the wildcard was not used. But since blurp.basjes.nl had no records in it, all you got back was an empty response. After that you added a CNAME record for blurp.basjes.nl, so when you asked yet again that's what you got back.

Does that clear things up?