2

We are having an issue with the dnscache service where its memory usage is becoming excessive (~6GB) after a week or two.

Restarting the service frees this memory but performing ipconfig /flushdns does not, an ipconfig /displaydns shows aprox 15-20 entries in the cache.

We have checked and there appears to be aprox 150 DNS queries per second taking place but I would not expect this to have the effect of causing this memory issue.

I have tried to search MSDN for hotfixes or bug reports but I could only find a reference to a memory leak in windows 2003. can anyone suggest how to proceed.

6
  • What is the "dnscache service"? I've never heard of it and can't find a service with that name on any of my W2K8R2 DNS servers.
    – joeqwerty
    Oct 4, 2012 at 12:53
  • its the DNS Client service the process is called DNSCache Oct 4, 2012 at 13:27
  • So this is a DNS client and not a DNS server? What is this client looking up 150 times a second?
    – joeqwerty
    Oct 4, 2012 at 13:30
  • the server is running an application server with 100's of connected clients, It looks like the app server is perfroming a lookup wheneach client sends data to the server. Oct 4, 2012 at 13:35
  • What is the version of dnsrslvr.dll?
    – Greg Askew
    Oct 4, 2012 at 21:08

4 Answers 4

2

Given that you’ve already searched for hotfixes, I assume you server is already up to date with the latest Windows Service Pack and current on all updates? That’s always the best place to start.

With a cache size of 6 GB, we’re talking tens of millions of unique DNS results (which is absurd). It’s unlikely that your clients would be making that many unique DNS queries. In the comments of your question, you indicated that the application is performing lookups each time a client sends data to the server. I don’t know how the application (I’m assuming web application) on your server works, but I wonder if some event is creating unique DNS queries with each client request, queries that get returned as a non-existent domains. I suppose that if the DNS query hits a domain with a wildcard, they might be returning valid responses. Either way, this could account for the huge cache size.

Ipconfig /displaydns does not always return the entire cache, which would explain why you’re only seeing 15-20 entries.

You can (at least I can) confirm this by performing the following…

From a Powershell prompt:

> ipconfig /flushdns
> [System.Net.Dns]::GetHostAddresses(“google.com”)

Check out the DNS cache

> ipconfig /displaydns

You’ll probably see the Google.com entry. Make note of the cache size using the task manager (in Task Manager, find the dnscache service, right click on it, select go to process).

Now run the following command, where ### is a large number (I used 10,000). Note, I set up my local DNS server to respond to all .test subdomains with a wildcard so I didn’t hit my ISPs DNS server 10,000 times in a short period of time.

> $i = 0
> do {[System.Net.Dns]::GetHostAddresses(“blahblahfakedomain” + $i++ + “.test”)} while ($i –lt ###)

As the command executes, watch the cache process’s memory utilization grow in Task Manager. Once it completes, check out the cache. You probably won’t see the ### quires we just generated, but the DNS Cache processes hasn’t released the memory.

In my experiments, I was able to recover the memory by flushing the DNS cache (10,000 queries is about 2 megs). I don’t know why that doesn’t work for you. Maybe because the cache is so darn big…?

Anyway, I propose the following workaround:

Modify the MaxCacheTtl and MaxNegativeCacheTtl registry values under the HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Dnscache\Parameters key per (http://support.microsoft.com/kb/318803):

Modify or create the DWord value MaxCacheTtl, setting it to something low, like 3600 seconds (0xE10 in hex). All values are specified in seconds. The default is 1 day (86,400 seconds).

Modify or create the DWord value MaxNegativeCacheTtl, setting it to 0.

This will cause all records to be cleared from the cache at most an hour after they are added. Negative results (non-existent domains) won’t be stored in the cache at all (I’ve confirmed this on my Windows 7 desktop).

I call this a workaround because I would say the problem lies with whatever process is creating the absurd amount of requests. If you can, try to quench the torrent of requests from the source.

3
  • Thanks for your help but we have found a work around see the answer I posted, but could you please take a look at the Wireshark trace to see if there is anything that could be causing this issue. Oct 5, 2012 at 8:47
  • @MikeT, I’m glad you were able to come up with a workaround. Unfortunately, the information provided in your new post isn’t enough (at least for me) to determine what the cause of the issue was. Can you provide any more information? For example, were the 150 DNS requests per second all identical to the one you’ve listed? Did your capture run on all interfaces? Sorry, I couldn't comment under your new post. Not enough points yet... Oct 5, 2012 at 19:09
  • all the DNS queries where identical to this one as far as I could see, i wasn't sure if the TTL of the cname being longer than that of the A record would cause the cache issues. incidentally now the memory usage of the cache is holding steady at 6 MB so it must be these queries that where causing the issue Oct 5, 2012 at 19:56
1

We have run a Wireshark capture to see what DNS queries where being run, and it turns out when each client sends data to the server, it is connecting to an MSSQL server on another machine and performing a DNS query to get the IP address of this SQL server.

As a work-around we have configured the application server with the SQL servers IP address so there is no need for the application server to query the DNS cache.

However the question still remains as to why, what appear to be normal DNS queries, are causing such memory usage in the DNS Cache.

I have included the wire shark trace here so that people may be able to help diagnose the issue

DNS Query

1

There are a lot of discussions about DNS high memory utilization on Windows 2008 R2 floating around. A lot of them point to disabling EDNS, which does not decrease the memory usage. The real problem is in fact due to the ports that DNS opens up: 2500 UDP IPv4 and 2500 UDP IPv6. This is a nice and round number for a busy production DNS server, however, for an internal DNS in a small office this is way too much.

To decrease the number of open ports, run the following command. For an internal DNS, with about 20 people in the office 100 ports is more than enough:

Dnscmd /Config /SocketPoolSize 100

Once the command executes, restart the DNS service; memory consumption should decrease.

You can also view how many ports you have open:

Dnscmd /Info /SocketPoolSize
1
  • Interesting i will take a look at this next time i speak to the client it was affecting. Aug 21, 2013 at 12:47
0

Not sure if this is way too late, but I stumbled into this while investigating a totally different issue - I think it is the .uk domain that was the cause of the issue.

They go on to describe that a simple regkey fix will resolve it.

http://support.microsoft.com/kb/968372

When name resolution is provided by root hints, Windows Server 2008 DNS and Windows Server 2008 R2 DNS Servers may fail to resolve queries for names in certain top-level domains. When this happens, the problem will continue until the DNS Server cache is cleared or the DNS Server service is restarted. The problem can be seen with domains like .co.uk, .cn, and .br, but is not limited to these domains.

When the problem is happening, an nslookup command issued for an affected name will return the error "server failed". A network trace will show that the DNS server does not send any traffic for such a request to the Internet. No events related to a problem are reported in the DNS Event Log.

This problem does not happen if DNS Server is configured to use forwarders for Internet name resolution instead of root hints.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .