Lire design: using network topology configuration files

[Arnaud Taddei]

Joost van Baal wrote:
> 
> [I reply to the list, since this discusses some general design issues.]
> 
> On Sat, Jan 26, 2002 at 06:37:19PM -0500, Francis J. Lacoste wrote:
> > On Fri, Jan 25, 2002 at 08:52:58AM +0100, Arnaud Taddei wrote:
> > >
> > >     IF the machine is a message store
> > >     THEN any message for user or user at localdomain which is in an
> > > SMTP-Accept line is a successful delivery to the user mailbox
> >
> > What does it mean if the machine isn't a message store? I'd like, if
> > possible, to be able to write DLF converters that don't need to be
> > informed of the topology of the network or the role of the machine to
> > work. This for several reasons:
> >
> > 1- If the logging system is reasonable that role can be inferred by the
> > log data. For example, from the sendmail's log file we know what is
> > local delivery, what is forward and what is an error, etc.
> >
> > 2- A DLF converter that needs to be configured (with the server's role)
> > before it can be used meaningfully cannot be use with the Online
> > Responder (where users sends their log files by email and get a
> > report back).
> >
> > 3- Lire doesn't yet have a centralized configuration mechanism (except
> > for the defaults file). This is something we will need for sure, but it
> > will take time to design and implement carefully (it's not rocket
> > science, but it would be good if such a major addition could be postpone
> > a little). What I have in mind, is a kind of API where modules
> > register configuration data they need. Having such a system has several
> > advantage: a generic user interface can be used; standardized
> > configuration procedure across the Lire components, automatic
> > documentation generation.
> >
> > I agree that with some knowledge of the topology better analyses can be
> > done. But those analyses are independent of the actual software in use.
> > Put another way: I think that analyses with knowledge of the topology
> > should be written at the superservice level (and thus available across
> > all services) rather than at the service level.
> >
> > In conclusion, I'd like, unless we can't find the appropriate heuristics
> > because of badly designed logging system, to have DLF converters that need to
> > be given the minimum amount of information to operate.
> 
> > > THIS MEANS that it is important that there is a configuration file that
> > > lire reads to understand which is the role of each machine name in the
> > > architecture. I will come back in a proposal mail that will follow.
> 
> > > So which is the lesson of this mail: We need to have a file that
> > > minimally describes the machines that are doing the service.
> >
> > Like I said above, I'd like DLF converters to contain proper heuristic
> > to be able to operate without such information. For the NMS case, I
> > think it is easy to determine if a server is a message store or a
> > hub/relay. In the message store case, there will be a lot of SMTP-Accept
> > records, few SMTP-Deliver. In the relay case, almost all SMTP-Accept will
> > have a matching SMTP-Deliver record.
> >
> > I do think that having information about the topology could provide us
> > with interesting analyses on the data, but I just think that such
> > analyses should be done in generic analyzers at the superservice level
> > (and thus orthogonal to the nms2dlf converter).
> 
> Indeed.  I agree: a dlf file ideally stores all information as found in
> the raw log.  Possibly less, e.g. in cases where different
> implementations of the superservice log different things.  Never more.
> If more information is needed (e.g. about network topology), this should
> be used only after the dlf has been written, while generating the
> reports.  (The file describing the roles of machines Arnaud is talking
> about is needed at this point.)  Such a setup might be very useful too
> when combining different reports: when doing combinations, knowledge
> about network topology is crucial.
> 
> However, if one cannot generate a sane email dlf file from the logs
> we're dealing with, some heuristics might be inevitable.  However, I'd
> prefer to avoid this.
We would hope we can avoid it, for sure, but real life might force us to
cover some exceptions. At least the notion of architecture_role
parameter can help us to rationalise on this concept the code. I would
propose a mix in the parser of:
- heuristic approach (needed because sometimes you cannot have an
architecture_role)
- the architecture_role approach as a parameter (because sometimes you
cannot afford relying your parsing on a heuristic, potentially for up to
legal reasons or business critical reasons)

My 0.02 euro

A++

> 
> Bye,
> 
> Joost
> 
> --
> Joost van Baal              . .           http://www.logreport.org/
>                            .   .
> /^LogReport$/               . .               joostvb at logreport.org
> 
>   ------------------------------------------------------------------------
>    Part 1.2Type: application/pgp-signature
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Arnaud.Taddei.vcf
Type: text/x-vcard
Size: 455 bytes
Desc: Card for Arnaud Taddei
Url : http://lists.logreport.org/pipermail/development/attachments/20020129/1d9b8321/attachment.vcf