1

Something I can't wrap my head around is happening with a DC on our network. I am a programmer by trade not a network engineer, but I am 99% sure this is not normal behavior.

Anyway when I rebooted the DC I had a cmd prompt open on my machine pinging the machine so I could see when it went down/came back up. However the pings never stopped, so I logged into the machine to look and see if it rebooted, which it did based on the system uptime being reported.

So I did the same thing with Linux running the ping command, which shows duplicate packets being returned on every request, except when the machine was down for the reboot.

So on windows the packets didn't change much when I rebooted it, just the TTL changed.

Reply from 10.0.0.254: bytes=32 time<1ms TTL=128
[... Machine rebooted]
Reply from 10.0.0.254: bytes=32 time<1ms TTL=255
[... Machine came back up]
Reply from 10.0.0.254: bytes=32 time<1ms TTL=128

On the Linux machine the duplicates stopped while the machine was down rebooting. Also every duplicate on the Linux ping has a TTL of 255 and the non-duplicates were all 128.

The machine in question is used for DC, DNS, and a file Server. It is listed as the second DNS server on both the machines tested.

If I ping another DC from both those machines there are no duplicates in Linux, and I suspect that it would not respond to pings while rebooting like the other machine.

Does anyone have any explanation what the heck is going on? The network has 2 AD DC/DNS servers with no load-balancing, clustering, or any other type of failover.

  • Take a look at your arp entries on your machine while it reboots. look to see if the Mac address changes. If you have access to the terminal on your switch, you can also look at Mac address tables there too. this will help you find if something is spoofing on your network. Is the DC a VM? – Linuxx Jan 7 '17 at 0:36
  • Both DCs are physical, and it is a very small network, there are less than 10 clients hooked to the domain. Spoofing chance is extremely unlikely. Anyway looking at the logs on the router (Linux based device) it looks like the mac address requesting the IP matches the correct MAC. I will try to look at ARP requests to see what is goin on there. – Brian D. Jan 7 '17 at 0:58
  • Something is responding during the reboot, so arp should shed some light. Keep in mind, spoofing isn't always malicious, it could be unintentional. Good luck! – Linuxx Jan 7 '17 at 1:00
  • P.S. Look at Joe's response below. Wireshark is your friend. – Linuxx Jan 7 '17 at 1:01
2

The default TTL for Windows (at least for all current versions) is 128, so those ping replies are from your DC. The ping replies with a TTL of 255 are from another device (probably Linux based) that has an A record in DNS that is the same as the A record for the DC. Take a look at the relevant DNS zone to find the offending A record and delete it. Check the system that corresponds to the offending A record to make sure it isn't configured with the same ip address as the DC.

  • Looking at DCs the DNS on both doesn't look out of the ordinary. Under Forward > domain.internal I have 1 entry for SOA || 2 entries for NS's, one for each DC || 2 entries for (same as parent), one for each DC || 2 entries for A, one for each DC || Normal other clients none with overlapping A records to the DCs – Brian D. Jan 7 '17 at 0:28
  • Run a packet capture on your machine and perform the same operation (ping the DC while rebooting it), then look at the source MAC address of the ping responses. Then use that information to track it down. – joeqwerty Jan 7 '17 at 0:55
  • So as a test I started a ping test on a Win and a Linux box. The Linux box was reporting dups and windows was not. This time I shutdown the DC. In both reported as unreachable. When I booted the server back up it was responding to pings normally for a short time then started showing dups again. Going to run wireshark next... – Brian D. Jan 7 '17 at 1:57
  • I ran wireshark with the DC on, and I see the ping request, then 2 responses. If I am reading this right both responses have identical source and destination MAC addresses. If I shutdown the DC it will show as unreachable. When I turn it back on I start getting responses before the OS loads. And then when it starts boots and the network comes up I will start getting duplicate responses. How is this even possible? – Brian D. Jan 7 '17 at 2:09
  • When you say that you're getting duplicate responses do you mean that you're getting one set of responses with a TTL of 128 and a second set of responses with a TTL of 255? If so, try to capture those in Wireshark and look at the source MAC address of both sets of responses. – joeqwerty Jan 7 '17 at 2:17
2

I was able to fix the issue by replacing the NIC in the server. I shut it down, installed a new NIC (different brand & model). I then booted into BIOS and disabled the other NIC which was onboard.

I then booted into Windows and made sure the IP address was correct (10.0.0.254). I then checked DNS to make sure that the DHCP address it pulled at boot was not listed in DNS. It was listed under (same as root) so I deleted that on both DCs. Cleared DNS cache and restarted the DNS service.

After that I pinged the machine from various other workstations and it is working fine now. So not sure if it was the MAC address changing, the driver changing, or the hardware changing.

Now just need to do a little research to see if I need to change anything for AD to work flawlessly since I changed NICs.

  • AD should be unaffected by the change. Did you say that the DC has a DHCP assigned ip address? If so, that's counter to best practice. – joeqwerty Jan 7 '17 at 3:14
  • 1
    Does the server have remote management capability? If so, does it share the onboard NIC? If so, that might explain the behavior. It may have been the remote management OS responding to the pings. That didn't occur to me until just now. – joeqwerty Jan 7 '17 at 3:16
  • Both the DCs are using DHCP reservation, so the new MAC address of the NIC caused it to pull a random IP. I could have set a static before swapping, or changed the MAC in the reservation but figured I would just clean up DNS quick after I fixed it. As far as remote management, it is an Intel motherboard, which I know some have IMEI. I don't use OOB management so not 100% sure but I will check it out. – Brian D. Jan 7 '17 at 3:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.