MailScanner snd SpamAssassin

[Wytze van der Raay]

Webmaster wrote:
> This may have been asked, but I have been unable to find an answer. 
> Sorry to bother you...
>  
> I currently use MailScanner which scans all incoming email for viruses
> and for spam via an external virus program and SpamAssassin.  I know
> lire can parse Spamassassin logs, but I have been unable to find a way
> to parse MailScanner logs.  Is this possible?  I use lire for all my
> other logs, it is great, and I am sure other people could use this.

Yes, this is definitely possible, but not out of the current Lire box.

> Currently, the only one I could find is SawMill, and I hate closed
> source.....  Below is a MailScanner excerpt:
>  
> Oct 19 00:09:40 scotty MailScanner[22221]: New Batch: Scanning 1
> messages, 1908 bytes
> Oct 19 00:09:40 scotty MailScanner[22221]: Spam Checks: Starting
> Oct 19 00:09:41 scotty MailScanner[22221]: Message 7B3D764C247 from
> 64.94.106.108 (usaplatinum at d-alertmail.com
> <mailto:usaplatinum at d-alertmail.com>) to perlone.com,nwdhosting.com is
> spam, SpamAssassin (score=17.807, required 6, BAYES_99 5.40, DCC_CHECK
> 2.91, HTML_IMAGE_ONLY_02 1.23, HTML_MESSAGE 0.10, HTML_WEB_BUGS 0.34,
> MIME_HTML_MOSTLY 1.24, RCVD_IN_BL_SPAMCOP_NET 1.50, RCVD_IN_NJABL 0.10,
> RCVD_IN_NJABL_SPAM 1.21, RCVD_IN_SBL 1.11, SUBJ_HAS_UNIQ_ID 2.68)
> Oct 19 00:09:41 scotty MailScanner[22221]: Spam Checks: Found 1 spam
> messages
> Oct 19 00:09:41 scotty MailScanner[22221]: Spam Actions: message
> 7B3D764C247 actions are deliver
> Oct 19 00:09:41 scotty MailScanner[22221]: Virus and Content Scanning:
> Starting
> Oct 19 00:09:41 scotty MailScanner[22221]: Uninfected: Delivered 1 messages
>
> I would love to see stats by user delivered to, top spam ip's, and
> grouped spam scores, infected mail, etc..  I hope this is possible.  I
> am a developer, if this has not already been done, can you can point me
> in the right direction?

The first thing needed to achieve this is a converter which extracts the
relevant information from the MailScanner logs and records it into
Lire's spamfilter DLF format. Lire 1.3 provides a nice mechanism for
converter plug-ins. To get you started, I have attached a fairly crude
converter which shows the full framework, and implements a primitive
parse of the MailScanner logs -- the meat of the module is in the
function process_log_file(). Your improvements on this will be very much
appreciated; please feed them back to this list or the
development at logreport.org list.

You can play with the converter by putting MailScannerConverter.pm in a
directory .lire/converters in your home directory and then trying:

	lr_run lr_log2report -o txt mailscanner <your_mailscanner_log

This will give you Lire's current default reports for the spamfilter
superservice.

You can modify the reports by looking at
/usr/share/lire/reports/spamfilter/* and /etc/lire/spamfilter.cfg, and
customizing these to your wishes. In this, you may be somewhat limited
by the current definition of the spamfilter DLF, which on one hand
defines a number of fields not logged by MailScanner (but you might be
able to extract some of them from surrounding sendmail loglines?), and
on the other hand has no provisions to store the detailed spaminfo like
DCC_CHECK, BAYES_99 etc.  So if you are really curious and adventurous,
you might want to extend or revamp the spamfilter DLF to accomodate
them. The Lire Developer's Manual should provide good guidance for that
( http://download.logreport.org/pub/current/doc/dev-manual/index.html ).

Please feed back any results you are getting to this list, so others can
benefit from it too, and better spamfilter reporting can be integrated
in subsequent Lire releases!

Regards,
Wytze van der Raay

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: MailScannerConverter.pm
Url: http://lists.logreport.org/pipermail/questions/attachments/20031021/5e21488d/attachment.pl