exim rejectlog with log snippets and new ideas, (was: Re: exim log analysis)

[Tom Northeast]

Sorry for double mailing, I sent the previous by accident!

Hi again group. I have been working on my problem for a while now, Joost and I had a discussion today and I thought I'd give everyone a run down of what I've found out etc.

> Basically our company runs two debian servers with exim, spamd and clamd.  The services upon these servers are sold to our customers as a no hassle
> solution to "pre-process" their corporate email for viri and spam.  The systems works on a round robin dns, so each server receives roughly an equal load of
> mail, this obviously generates two separate log files.
>
> Because all the customers accounts pass through the same servers, I cannot simply generate statistics for "all email" I have to sort it per receiving domain 
> so I can show each company virus and spam statistics for their own email. This isn't helped by the fact that I have two separate log files to parse!

So I set about finding my way around Lire, how it works, previous peoples experiences and testing its various abilities with the files I had to work with.

I checked out Lire's spamassassin2dlf converter, unfortunately my spamd logs are unusable as the daemon has been logging in debug mode and it just wouldn't give me the output I required.

Concentrating on the exim logs themselves, I found out today that the mainlog does log when an incoming message is stopped due to either spam or Viri BUT it does not log the recipients address!! eg.

2005-02-02 14:04:01 SMTP connection from [206.158.x.x] I=[193.108.x.x]:25 (TCP/IP connection count = 1)
2005-02-02 14:04:02 1CwL6w-xxxxxxx-A5 H=crc2.xxxxx.us (mail01.xxxxxx.us) [206.158.x.x] I=[193.108.x.x]:25 F=<tester at testvirus.org> rejected after DATA: Virus/Trojan detected in this email (Eicar-Test-Signature)
2005-02-02 14:04:02 SMTP connection from crc2.xxxxxx.us (mail01.xxxxxx.us) [206.158.x.x] I=[193.108.x.x]:25 closed by QUIT

The output is the same for spam and the score is given. But no recipient address! so the mainlog is of no use to me.


Then Joost and I discovered that in Exim's Rejectlog, after the timestamped line containing the rejection reason, the messages header is dumped to the log file. On the first line of the header is the email's Envelope entry. eg.

2005-02-02 14:04:02 1CwL6w-xxxxx-A5 H=crc2.xxxxx.us (mail01.xxxxx.us) [206.158.x.x] I=[193.108.x.x]:25 F=<tester at testvirus.org> rejected after DATA: Virus/Trojan detected in this email (Eicar-Test-Signature)
Envelope-from: <tester at testvirus.org>
Envelope-to: <xxxxx at xxxxxxx.com> {this is the part I require)

I think that this is now the only realistic option I have for producing the analysed results I require. If anyone could help me getting started with a DLF converter, I would be more than happy to assist with the development of it, as joost said in Irc:

[15:43] <@joostvb> i guess lots of people are doing this kinda stuff: spam and virus rejecting stuff
[15:43] <@joostvb> it might be possible to define a new superservice for that

Thanks!!!!

Tom

-- 
To UNSUBSCRIBE, email to questions-request at logreport.org with a subject of 
"unsubscribe". Trouble? Send an email with subject "help" to 
questions-request at logreport.org