Interpreting exim log files after parsing

Question

I'm parsing exim log files and, due to my processing method, lose the original order of all entries in this file. I rebuild the transactions by their transaction ID (i.e. 1OfiYX-0000Ev-7k) but still don't have a way to determine the original order.

The <= , => , == , ** characters' original order matter, right? Is there a way to rebuild the order without any additional information?

Αντίο

Answer 1

I think the answer, limited solely to the symbols you ask about, is that <= is always going to come before each of the other symbols you list, and a message's Completed line will come after all of those symbols. However, each of the other symbols ==, =>, **, can appear in any order between <= and Completed.

One thing to keep in mind is that a message can have multiple recipients, and each of those recipients can be deferred (==), so the order of those symbols matters for each recipient of the message

So, every message should have exactly one <= when the message is accepted by the local server

Every message should have exactly one Completed line indicating that the local server is done with the message

Between those:

Each message:recipient will have exactly one of ** (failure) or => (delivered). It will be the last entry for that specific message:recipient.

Each message:recipient may have one or more == (deferred) lines. If a message:recipient has a == log line, it will occur before that message:recipient's => or ** line.

The order of different recipients in a given message only matters if you think it matters, most likely.

Answer 2

Exim is shipped with tools to help with logfile analysis. In particular, exigrep may be of interest, as it can search for a pattern in a line and then show all the log-lines for that message, including those which came before the match-line.

Exim is shipped with documentation, "The Exim Specification"; at the very least, you should have a file called "spec.txt", if not .pdf or other variant; this is also online at http://www.exim.org/; you might find "49. Log files", documenting the precise format of the log-files, and "50. Exim utilities" to be useful.

Each log-line has a timestamp; group by exim message-id and then sort by timestamp and you have the original order back.

Answer 3

Yes the matter as they indicate the direction of the message flow. You need to improve your processing method not to reorder your entries.

Answer 4

Have you tried using one of the available log file parsers, e.g. awstats or sawmill.