Lire design: using network topology configuration files

[Joost van Baal]

[I reply to the list, since this discusses some general design issues.]

On Sat, Jan 26, 2002 at 06:37:19PM -0500, Francis J. Lacoste wrote:
> On Fri, Jan 25, 2002 at 08:52:58AM +0100, Arnaud Taddei wrote:
> > 
> > 	IF the machine is a message store 
> > 	THEN any message for user or user at localdomain which is in an
> > SMTP-Accept line is a successful delivery to the user mailbox
> 
> What does it mean if the machine isn't a message store? I'd like, if
> possible, to be able to write DLF converters that don't need to be
> informed of the topology of the network or the role of the machine to
> work. This for several reasons:
> 
> 1- If the logging system is reasonable that role can be inferred by the
> log data. For example, from the sendmail's log file we know what is
> local delivery, what is forward and what is an error, etc.
> 
> 2- A DLF converter that needs to be configured (with the server's role)
> before it can be used meaningfully cannot be use with the Online
> Responder (where users sends their log files by email and get a
> report back). 
> 
> 3- Lire doesn't yet have a centralized configuration mechanism (except
> for the defaults file). This is something we will need for sure, but it
> will take time to design and implement carefully (it's not rocket
> science, but it would be good if such a major addition could be postpone
> a little). What I have in mind, is a kind of API where modules 
> register configuration data they need. Having such a system has several
> advantage: a generic user interface can be used; standardized
> configuration procedure across the Lire components, automatic
> documentation generation.
> 
> I agree that with some knowledge of the topology better analyses can be
> done. But those analyses are independent of the actual software in use.
> Put another way: I think that analyses with knowledge of the topology
> should be written at the superservice level (and thus available across
> all services) rather than at the service level. 
> 
> In conclusion, I'd like, unless we can't find the appropriate heuristics 
> because of badly designed logging system, to have DLF converters that need to 
> be given the minimum amount of information to operate.

> > THIS MEANS that it is important that there is a configuration file that
> > lire reads to understand which is the role of each machine name in the
> > architecture. I will come back in a proposal mail that will follow.

> > So which is the lesson of this mail: We need to have a file that
> > minimally describes the machines that are doing the service.
> 
> Like I said above, I'd like DLF converters to contain proper heuristic
> to be able to operate without such information. For the NMS case, I
> think it is easy to determine if a server is a message store or a
> hub/relay. In the message store case, there will be a lot of SMTP-Accept
> records, few SMTP-Deliver. In the relay case, almost all SMTP-Accept will
> have a matching SMTP-Deliver record. 
> 
> I do think that having information about the topology could provide us
> with interesting analyses on the data, but I just think that such
> analyses should be done in generic analyzers at the superservice level
> (and thus orthogonal to the nms2dlf converter).

Indeed.  I agree: a dlf file ideally stores all information as found in
the raw log.  Possibly less, e.g. in cases where different
implementations of the superservice log different things.  Never more.
If more information is needed (e.g. about network topology), this should
be used only after the dlf has been written, while generating the
reports.  (The file describing the roles of machines Arnaud is talking
about is needed at this point.)  Such a setup might be very useful too
when combining different reports: when doing combinations, knowledge
about network topology is crucial.

However, if one cannot generate a sane email dlf file from the logs
we're dealing with, some heuristics might be inevitable.  However, I'd
prefer to avoid this.

Bye,

Joost

-- 
Joost van Baal              . .           http://www.logreport.org/
                           .   .
/^LogReport$/               . .               joostvb at logreport.org

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://lists.logreport.org/pipermail/development/attachments/20020127/2742edfb/attachment.bin