2

After rebuilding our Exchange 2003 server due to hard drive failure, Exchange becomes unavailable after an unknown amount of time. Vast majority of the users are now on Outlook 2007 (handful still on 2003).

We turned up the logging level and we've noticed these warnings now; I've been tracking them and it looks like they occur every 15 minutes or so (not sure if the time frame has anything to do with it)

These warnings sometimes lead to errors and yesterday we rebooted the server twice; around 8:30am and 2:30pm (about 6 hours - again, not sure if timing has anything to do with it)

Process INETINFO.EXE (PID=1300). DSAccess needs to close a connection to the Domain Controller AD-server.domain.com due to error 0x80040951.

Process STORE.EXE (PID=2936). DSAccess needs to close a connection to the Domain Controller AD-server.domain.com due to error 0x80040952.

Process MAD.EXE (PID=2160). DSAccess needs to close a connection to the Domain Controller AD-server.domain.com due to error 0x80040952.

each of the 3 warnings reoccur every 15 min or so.

The errors that preceed us rebooting exchange look like

LDAP Bind was unsuccessful on directory AD-server.domain.com for distinguished name ''. Directory returned error:[0x51] Server Down. DC=domain,DC=com

We're not sure if the warnings have anything to do with the errors (and ultimately, the reboot that follows). By the timing above, we thought it would happen again around 8:30pm and 2:30am but nothing. No errors since the reboot around 2:30pm yesterday.

I should note Exchange is in DMZ 1 and AD is in DMZ 3 but the firewall (Sonicwall) is completely open between the 2 DMZ's.

Exchange used to be on its own server but per recommendation from some techs, we've since moved it to a VM. 2008 server is the host, VMWare server 2 for the VM and Windows 2003 running 2003 Exchange.

We're really at a loss as to whats happening. We've rebooted the firewall, turned off AV / Content Filtering, rebooted both AD and Exchange.

We're considering moving Exchange into the same DMZ as AD. Everyone we speak to recommends that, but we can't do that just yet. The best part is, until we had to rebuild Exchange, we ran perfectly fine, same firewall, same DMZ setup, same OS versions (except for the VM) for 3+ years. Only after we lost Exchange and had to rebuild it have we been having these problems.

Any ideas?

--- notes added 11-23-11 11:11am EST --- @ Even Anderson

I wasn't entirely sure how to accomplish what you were asking me to do; we don't normally sniff traffic here...

Then I remembered our Sonicwall NSA has built in packet capturing capabilities.

So I inputted the exchange server and AD servers, had it send the captures to an FTP server on my PC and I can now watch the traffic between exchange and AD. It sends me .cap files which I'm viewing with Wireshark.

The "problem" happened this morning between 1 and 3am, and then again this morning around 9am. I rebooted when I came in around 6am just to be on the safe side and rebooted again when exchange became unresponsive around 9:30am.

Filtering for LDAP protocol, I'm seeing the following entries:

SASL GSS-API Inegrity - these look like the actual lookups and every searchRequest has a searchResEntry

bindRequest and bindResponse - these look 1 to 1 - so that looks fine.

I am seeing some unbindRequest from Exchange to AD with what appears to be no response - I'm not sure if it should have a response though.

I'm not seeing anything with an actual SYNC in it.

Still looking - Running the capture doesn't seam to be effecting performance anywhere so I hope to continue to run it until the errors occur and exchange stops responding.

1 Answer 1

2

The error 0x80040951 is an "LDAP_SERVER_DOWN" error, and the 0x80040952 is an "LDAP_LOCAL_ERROR". Both of these make me think that the Exchange Server computer is becoming unable to communicate with the DC via LDAP.

I'd start by sniffing the traffic between Exchange and the DC. If you can, use a SPAN port or other method to "tee" in a dedicated sniffer computer between the the DC and Exchange so that your long-term sniffing can run w/o causing a performance problem on either server. You could use a capture filter to isolate the traffic to the conversations between the Exchange Server and the DCs. If you're able, use two computers to sniff and "tee" one in between each server and the firewall device isolating them.

Without seeing what's happening with the actual network traffic it's difficult for me to make any further recommendations. If you end up with some captures of the failure "in the act" you can post them up here and we'll have a look. Based on what I'm seeing so far I expect you to discover the Exchange Server computer repeatedly SYN'ing for the LDAP port on the DC and not getting a response. If you're lucky enough to be able to capture from both "sides" of the firewall device I bet you'll see traffic that isn't traversing the firewall. Your problem certainly has that feel to it...

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .