The Setup
We are running exim version 4.72 on Debian Squeeze 6 (Squeeze) (LTS).
Exim is used for outgoing emails only - it does not handle emails from other servers/domains - it only handles (outgoing) message from a local running java (web) application.
For anonymization, lets assume the domain we use for our company email addresses, as well for the domain our java web application is running on, is called example.com.
The Background
We have a web application written in a java framework which allows it's users to send out emails. The envelope-from
address of these emails belongs to our domain: [email protected].
Technically these emails are created and get send from the java web application (we use the JavaMail API) locally to exim, which then eventually sends them out to their destination address.
Incoming emails to our domain are handled by Google Apps/Gmail (We also set up SPF records, etc.)
This setup is working fine without any problems so far.
For completion, here is our (anonymised) update-exim4.conf.conf
:
root@server:~# cat /etc/exim4/update-exim4.conf.conf
# ... comments are skipped ...
dc_eximconfig_configtype='internet'
dc_other_hostnames='node1.example.com'
dc_local_interfaces='127.0.0.1 ; ::1'
dc_readhost=''
dc_relay_domains=''
dc_minimaldns='false'
dc_relay_nets=''
dc_smarthost=''
CFILEMODE='644'
dc_use_split_config='true'
dc_hide_mailname=''
dc_mailname_in_oh='true'
dc_localdelivery='maildir_home'
The Problem
Recently we are receiving bounce emails because users of our java web application tried to send out emails to non-existing email addresses.
Lets have a look on such a case in the (anonymised) log file:
root@server:~# cat /var/log/exim4/mainlog
# try to send an email to an unrouteable address
2014-06-01 17:34:07 1Wr7lv-0000CR-G0 <= [email protected] H=localhost (node1.example.com) [127.0.0.1] P=esmtp S=620363 id=484648301.663.1401636847496.JavaMail.webapp@node1
2014-06-01 17:34:07 1Wr7lv-0000CR-G0 ** [email protected]: Unrouteable address
2014-06-01 17:34:07 1Wr7lv-0000CR-G0 Completed
# Immediately a bounce message is generated and gets send to the the origin sender
2014-06-01 17:34:07 1Wr7lv-0000CU-OZ <= <> R=1Wr7lv-0000CR-G0 U=Debian-exim P=local S=108065
2014-06-01 17:34:09 1Wr7lv-0000CU-OZ => [email protected] R=dnslookup T=remote_smtp H=aspmx.l.google.com [74.125.136.26] X=TLS1.0:RSA_ARCFOUR_SHA1:16 DN="C=US,ST=California,L=Mountain View,O=Google Inc,CN=mx.google.com"
2014-06-01 17:34:09 1Wr7lv-0000CU-OZ Completed
What we want to achieve
We now want to show (or notify - whatever) each java web application user within the java web application that an email which he/she wanted to send actually failed to sent.
(Remember: The bounce email are currently sent back to [email protected] - not to the web application users email address - so these users have no clue their emails did not get send)
The basic (and simple) idea we now have:
Make exim write the id
of each failed message (which in the above case is 484648301.663.1401636847496.JavaMail.webapp@node1) into a local running PostgreSQL database as soon as possible we know the message delivery failed (or: as soon as possible the corresponding bounce message was generated and/or sent).
If this is possible, we could easily read these "failing" id
s from the database within our java web application, and - voilà! - we can show users that their messages have been bounced.
Simple idea, isn't it?
Just for a better understanding: The mentioned (unique) id
s already get generated for each message within the java web application before the JavaMail API sends the message to exim - so we can be sure our java web application knows each id
for each message (and of course also knows which user sent out a specific message).
Possible Solutions?
I did some research on how this could be solved and this is with what I came up with by now (Please correct my if I am wrong - these are just my thoughts right now):
Approach 1: Pipe the failed message and it's id
to a bash script. Such a bash script could then easily store the id
in the local PostgreSQL database. Is this possible and how could this be achieved? The problem I see is that as soon a message failed, no other exim router gets executed - which means I can not write a router which handles a failed message - the failed message would never reach the router. Or am I wrong?
Approach 2: Another possibility I was thinking about is to not have a look at the original message which failed, but the newly generated bounce message. As far as I understand this newly bounce message is also going through all routers until one accepts it. So maybe we could write a router which checks if a message is a bounce message and then pipe it to the bash script? But how do I know that if a message is a bounce message? Does it have a special header or something else significant? (Remember: As we only use exim for outgoing emails we can be sure this would always be a bounce message "generated" by our java web application users).
Approach 3: Similiar to approach 2, but instead of writing a router we could make use of the system filter, which looks after a bounce message and then executes a command / pipe...
Approach 4: Would it be possible to configure error_copy
or even error_to
to not be an email address but a pipe to shell script (which of course received the id and/or whole failed message)?
Two more things: It seems exim can also talk directly to databases (I read this somewhere) - but I have no idea how I could make use of this for my described problem.
Plus: However this is solved, I do not want to modify any configuration files which gets overriden when exim gets updated via apt-get
(I do not want to re-configure exim after each update).
You help is needed
How would you solve this problem (saving the failed message id
s into the database)?
Please correct me If I was wrong somewhere - I am not that deep into the exim configuration. Also, as you can see, the solutions I think about are more theoretical - it would be nice I someone could show me real world configurations and where to put this configs to make them work.
Thank you very very much for any help!