We have a Windows 2003 Active Directory domain called 'domain.com', which also happens to be the domain name of our website. Recently we switched to using our AD servers as our DNS servers but have run into a little snag. Boxes that were using their own SMTP servers now cannot send out mail because AD doesn't contain a DNS record according to 'dig':
[user@hostname]$ dig MX domain.com
; <<>> DiG 9.4.2-P2 <<>> MX domain.com
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 41859
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0
;; QUESTION SECTION:
;domain.com. IN MX
;; AUTHORITY SECTION:
domain.com. 3600 IN SOA win2003adserver.domain.com. hostmaster.domain.com. 6168 900 600 86400 3600
;; Query time: 3 msec
;; SERVER: 10.10.10.27#53(10.10.10.27)
;; WHEN: Mon Jul 27 12:42:44 2009
;; MSG SIZE rcvd: 98
So what it's trying to do is, since there is no valid MX record, send mail through the primary AD server which isn't running SMTP. I should get back an MX record for mail.domain.com with a valid IP for the mail server.
What is the proper way to add an MX record in this situation? According to our Windows admin there is an MX record in the DNS for AD, but according to the above its not relaying it to anyone.
Solution
In this case the AD servers were doing exactly what they were told to do as there was no MX record entered for 'domain.com' that pointed to our mail server (Our AD Admin had entered in an MX record for the subdomain of 'mail.domain.com' instead of 'domain.com' as he didn't read the instructions on the screen for entering a new MX record).
I deleted the MX record and added a new one correctly and everything started working.