Two identically configured sendmails, one of them is not delivering mail

Question

I have two identically configured OpenBSD machines that use sendmail for mail delivery. I basically made no changes to the sendmail install other than setting up an alias root: systems@example.com on each one. They're both on the same subnet(s) and both can access the mail server mail.example.com. However, one of them cannot deliver mail. I'm not terribly familiar with sendmail, so hopefully someone can explain this behaviour. I've posted the /var/log/maillog/ from each machine below:

The machine which delivers successfully:

Dec  7 11:57:35 gw003 sendmail[9640]: nB7JvZmS009640: from=kamil, size=60, class=0, nrcpts=1, msgid=<200912071957.nB7JvZmS009640@gw00
3.backbone.example.com>, relay=root@localhost
Dec  7 11:57:35 gw003 sm-mta[30258]: nB7JvZlR030258: from=<kamil@gw003.backbone.example.com>, size=401, class=0, nrcpts=1, msgid=<2
00912071957.nB7JvZmS009640@gw003.backbone.example.com>, proto=ESMTP, daemon=MTA, relay=smmsp@localhost.backbone.example.com [127.
0.0.1]
Dec  7 11:57:35 gw003 sendmail[9640]: nB7JvZmS009640: to=systems@example.com, ctladdr=kamil (1031/1031), delay=00:00:00, xdelay=00:
00:00, mailer=relay, pri=30060, relay=[127.0.0.1] [127.0.0.1], dsn=2.0.0, stat=Sent (nB7JvZlR030258 Message accepted for delivery)
Dec  7 11:57:35 gw003 sm-mta[29848]: STARTTLS=client, relay=mail.example.com., version=TLSv1/SSLv3, verify=FAIL, cipher=AES256-SHA,
 bits=256/256
Dec  7 11:57:35 gw003 sm-mta[29848]: nB7JvZlR030258: to=<systems@example.com>, ctladdr=<kamil@gw003.backbone.example.com> (1031/1
031), delay=00:00:00, xdelay=00:00:00, mailer=esmtp, pri=30401, relay=mail.example.com. [10.2.32.11], dsn=2.0.0, stat=Sent (4b1d5e2
f-0000528c Message accepted for delivery)

The machine which is failing to deliver:

Dec  7 12:00:35 gw004 sendmail[12957]: nB7K0ZdA012957: from=kamil, size=60, class=0, nrcpts=1, msgid=<200912072000.nB7K0ZdA012957@gw0
04.backbone.example.com>, relay=root@localhost
Dec  7 12:00:35 gw004 sm-mta[25655]: nB7K0Z7U025655: <systems@example.com>... User unknown
Dec  7 12:00:35 gw004 sendmail[12957]: nB7K0ZdA012957: to=systems@example.com, ctladdr=kamil (1031/1031), delay=00:00:00, xdelay=00
:00:00, mailer=relay, pri=30060, relay=[127.0.0.1] [127.0.0.1], dsn=5.1.1, stat=User unknown
Dec  7 12:00:35 gw004 sm-mta[25655]: nB7K0Z7U025655: from=<kamil@gw004.backbone.example.com>, size=60, class=0, nrcpts=0, proto=ESM
TP, daemon=MTA, relay=smmsp@localhost.backbone.example.com [127.0.0.1]
Dec  7 12:00:35 gw004 sendmail[12957]: nB7K0ZdA012957: nB7K0ZdB012957: DSN: User unknown
Dec  7 12:00:36 gw004 sm-mta[25655]: nB7K0Z7W025655: from=<>, size=2203, class=0, nrcpts=1, msgid=<200912072000.nB7K0ZdB012957@gw004.
backbone.example.com>, proto=ESMTP, daemon=MTA, relay=smmsp@localhost.backbone.example.com [127.0.0.1]
Dec  7 12:00:36 gw004 sendmail[12957]: nB7K0ZdB012957: to=kamil, delay=00:00:01, xdelay=00:00:01, mailer=relay, pri=31084, relay=[127
.0.0.1] [127.0.0.1], dsn=2.0.0, stat=Sent (nB7K0Z7W025655 Message accepted for delivery)
Dec  7 12:00:36 gw004 sm-mta[9947]: nB7K0Z7W025655: to=<kamil@gw004.backbone.example.com>, delay=00:00:01, xdelay=00:00:00, mailer=
local, pri=32465, dsn=2.0.0, stat=Sent

Answer 1

I suspect a name resolution problem on the "bad" system. Check /etc/hosts for non-symmetric entries. Check DNS resolution on both machines.

ifconfig -a            # note various IP adresses of this system
dig -x 192.1.2.3       # query FQDN for my own ip, are results symmetric?
dig -x 1.2.3.4         # same on another interface
dig -x 5.6.7.8         # and another one

and

dig my.fully.qualified.domain.name.
dig another.fully.qualified.domain.name.

and

hostname   # is it symmetric on both systems? or short on one and fully-qualified on another?

Answer 2

relay=smmsp@localhost.backbone.example.com [127.0.0.1]

You are not relaying through your mail server mail.example.com

This is certainly a configuration problem related to relay domain or the server domain. (on the second machine it delivers mails on localhost, meaning it believes that the final destination is itself. check domain/hostname/relays)

Answer 3

Check the contents of /etc/mail/local-host-names and /etc/mail/relay-domains and maybe /etc/mail/mailertable -- your second box needs to know either it is example.com, or how to get there.