RFC: new datastructure for email 2dlf's

[Francis J. Lacoste]

On Wed, Mar 06, 2002 at 04:34:48PM +0100, Joost van Baal wrote:
> Hi,
> 
> I believe I've found a bug in sendmail2dlf:
> 
> Mar 15 13:34:05 firewall sendmail[277]: NAA00277: 
>  from=<john.doe.1 at 1.mail.example.com>, size=1943, class=-30, pri=85943, 
>  nrcpts=1, msgid=<john.doe.2 at 2.mail.example.com>, proto=ESMTP, 
>  relay=1.example.com.nl [10.0.0.1]
> Mar 15 13:34:05 firewall sendmail[277]: NAA00277: 
>  to=<john.doe.3 at 3.mail.example.com>, delay=00:00:00, mailer=smtp, stat=queued
> Mar 15 13:39:52 firewall sendmail[401]: NAA00277: 
>  to=<john.doe.3 at 3.mail.example.com>, delay=00:05:47, xdelay=00:00:02, 
>  mailer=smtp, relay=2.example.com.nl. [10.0.0.2], stat=Sent (Mail accepted)
> 
> gets converted to
> 
> 1016195645 firewall NAA00277 <john.doe.2 at 2.mail.example.com> john.doe.1 
>  1.mail.example.com 1.example.com.nl 10.0.0.1 1943 347 0 john.doe.3 
>  3.mail.example.com 2.example.com.nl 10.0.0.2 sent (mail_accepted)
> 
> I feel there should be 2 dlf records generated.  The delivery-attempt with
> stat=queued and delay=00:00:00 should occur in the dlf too.

IIRC, this was by design (this "bug" is present in all email DLF converters). 
For the email superservice, one DLF record represent one "final"
delivery. It shouldn't show the intermediary steps to the final delivery
(not counting forwards).  The only deffered records that should appear in the 
DLF file are those that weren't delivered at the end of the log file. If we 
start adding "intermediary" events (like the deferred in this case). We might 
as well add all of them and redesign the email superservice like it was
proposed by Arnaud Taddei around a multiple events DFL record. All email
events (pickup, delivery, filtering, errors) would then be represented
in the DLF. I don't see why we should have deferred events but not the
others.

> 
> Another tricky thing is: in some to= lines, more than one to_user
> occurs.  All to_user's are handled by the same to_relay_host in such a
> situation.  E.g.:
> 
>  Mar 15 14:00:02 mailhost sendmail[624]: NAA00594: 
>   to=john.doe.1 at 1.mail.example.com,john.doe.2 at 1.mail.example.com, 
>   delay=00:06:25, xdelay=00:00:01, mailer=smtp, relay=1.example.com.nl. 
>   [10.0.0.1], stat=Sent (OK)
> 
> This would mean the sendmail2dlf datastructure should get changed.
> 
> Currently, we have this:
> 
> -------
> 
> %msg stores all currently being processed information from the log.  This
> hash is indexed by dlfid's:
> 
>  $dlfid = $log->{'hostname'} . $log->{'queueid'}
> 
> Scalar values:
> 
>  $msg{$dlfid} = {
>    logrelay        => ...,
>    queueid         => ...,
>    time            => ...,
>    nrcpts          => ...,
>    from_user       => ...,
>    from_domain     => ...,
>    from_relay_host => ...,
>    from_relay_ip   => ...,
>    size            => ...,
>    msgid           => ...,
>  };
> 
> Furthermore:
> 
>  $del = $msg{$dlfid}->{deliveries}{$to};
> 
> $to is an email address as returned by &Lire::Email::sanitize_tos .
> 
>  $del->{to_user}   = ....;
>  $del->{to_domain} = ....;
> 
> Other $del keys with scalar values:
> 
>  to_relay_host
>  to_relay_ip
>  delay
>  xdelay
>  stat
>  xstat
> 
> -------
> 
> I propose a datastructure like this:
> 
>  $dels = $msg{$dlfid}->{deliveries};
> 
>  $dels = [
>    {
>      to_relay_host => ...,
>      to_relay_ip   => ...,
>      delay         => ...,
>      xdelay        => ...,
>      stat          => ...,
>      xstat         => ...,
>      to_users      => [
>        {
>          to_user   => ...,
>          to_domain => ...,
>        },
>        {
>          to_user   => ...,
>          to_domain => ...,
>        },
>        ...
>      ],
>    },
>    {
>      to_relay_host => ...,
>      ....
>      to_users      => [ ... ],
>    },
>    ...
>  ];
> 
> 
> 
> What do you think?
> 

Yes, that is possible. That way, it will be clear that the relay_host
was shared between recipients and was part of the same delivery. Altough
this information will be lost in the generated DLF (the fact that they were 
part of the same delivery). The other small benefit will be that we will save 
a few keys (to_relay_host, to_relay_ip, etc.) that were duplicated across all 
the to_users hashes in the old structure.

-- 
Francis J. Lacoste
francis at Contre.COM
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 232 bytes
Desc: not available
Url : http://lists.logreport.org/pipermail/development/attachments/20020306/f9ebc6dd/attachment.bin