# Extract all messages in a set of maildirs with certain domains in the headers

I have a mailserver (running Exim as MTA/MDA, and Dovecot for IMAP access) with around 50 users, and around 100GB total data (including some huge accounts, some defunct ones, etc). The mail is all stored in Maildirs. We have a sudden need to pull out all mails where the headers (To, From, Cc, etc) contain one of a handful of domains, to satisfy a request from our lawyers.

Now, I can hack together an inefficient solution (grep -R through the mail archive for the domains in question, a touch of cut, sort and uniq to get just the distinct filenames, copy all those files to a new Maildir and take it from there, perhaps) but this is going to take a hell of a long time to run on the hardware available. Is there a tool out there that will take away the pain of this process for me?

Platform isn't a huge issue - the server in question runs Ubuntu 12.04, but I have a sufficiently recent snapshot of the data I can mount on a machine running anything reasonable - and there's no requirement that the solution be FOSS, although the indicative software budget is in the hundreds not thousands of pounds.

I suspect there's a really obvious answer to this that Google isn't showing me, probably because I have the wrong search terms - anyone have an experience of this?

Thanks!

-

grep is indeed the tool for this job:

grep -Rli "^to: .*@yourdomain" /your/maildir/ | sort | uniq -u


Will yield a sorted list of unique filenames.

The reason you did not get results from .dotted directories is that "grep *" expands to all files and directories not starting with a dot; this has nothing to do with grep. Instead, pass the name of the upper maildir directory to grep and let it do the file searching: grep -R /your/maildir/

EDIT: not all grep binaries have -P compiled-in, but you don't need it in this case, so I removed it.

-
Thanks for the enhanced grep invocation, that simplifies things :-) My misdiagnosis of how -R was behaving with .dotted directories was actually even dumber than being confused by globbing -- I inferred it from the number of results I was seeing, and realised I was only grepping for one domain of the three, and in a user's Maildir who wouldn't have dealt with people in that specific domain much at all - doh! –  James Green Oct 23 '12 at 13:19