Help understanding some Lire conceptual issues

[Brad Knowles]

At 11:20 AM -0600 2004/03/04, Jim Lancaster wrote:

>  Am I to assume that Lire, itself, provides no mechanism for
>  collecting/accumulating/storing the log files to be processed in the
>  standalone configuration?  Does Lire leave it up to you to get the log
>  files to the server by whatever method?

	Pretty much.  There are some additional tools grafted on top to 
allow the processing of some log data by e-mail, but that's a front 
end to lire itself.  Moreover, since it is impractical to e-mail 
large files (and you have to do something with the output), IMO 
that's of limited benefit.

	The real use for lire is to take log data that has been collected 
centrally, or where the files have been moved to a central location, 
and then generate summaries for that input in text and/or HTML and/or 
PDF formats.

>  Completely off-topic:  My operations manager is from Brussels. He came
>  to the states as high school exchange student and never left.  One of
>  our best clients is a South African.  When he met Serge for the first
>  time, he noticed a trace of an accent--actually he noticed the *lack* of
>  an accent--and asked Serge where he was from.  Serge says, "Would you
>  believe East Texas?" (a region of almost unintelligibly-thick accents,
>  and obviously the wrong answer).

	I lived in central Oklahoma for much of my childhood, the rest of 
which was spent in places like Manhattan Kansas, Knoxville & 
Murfreesboro Tennessee, and Wilmington North Carolina.  Most of my 
family is in the west to middle Tennesse region.  My wife and her 
family are from San Antonio.  So, I am intimately familiar with 
accents.

	That said, I have been told that I have relatively little in the 
way of accent myself.  But let me go back to Tennesseefor a week, and 
I'm almost as bad as they are.

>                                    Julian didn't fall for it, so Serge
>  tells him the real story.  Next thing I know, the two of them are
>  jabbering away in Afrikaans, Julian's native language.  I had no idea
>  that Afrikaans and Flemish were so closely related.

	South Africa was settled by the Dutch (and was a Dutch colony for 
a very long time), from which Afrikaans is derived.  Flemish and 
Dutch are very closely related, but native Flemish speakers will tell 
you that there are many different dialects of each, and most can tell 
pretty much exactly where another native speaker comes from, by their 
accent and the way they phrase certain things.

	I used to work for a Dutch consulting company, and I was 
continually trying to get them to give me language lessons in proper 
Dutch down here in Brussels (as opposed to Flemish), and they were 
never able to find someone who was able to do that.

>                                                       It was like
>  old-home week.  Your name surname appears to be neither Flemish or
>  French--Are you English? (Please ignore the question if I am being too
>  personal.)

	Typical American mutt.  I have ancestors that are Black Irish, 
Native Americans, British, and pretty much the whole gamut.  My wife 
is the same, adding in strong German and French influences, and on 
one side is able to trace her family lineage all the way back one of 
to the soldiers who created the first permanent military encampment 
in what is now Texas (just outside of what is now San Antonio), about 
thirty years before the first official European (well, Spanish) 
settlers arrived.  Going to a Texas or San Antonio history museum 
with them is almost like visiting a family picture album.

>  If I might ask (especially after the last question <g>), what is your
>  general configuration?  Do you have Lire running on the same box as the
>  services generating the logs?  Or are you using some method to move the
>  logs to the Lire server?  Do you have multiple Lire servers?

	I am not currently employed, although I am working on setting up 
my own consulting company, and I also have a book idea I'm working 
on, as well as a booklet idea for SAGE (the System Administrators 
Guild), and I may decide to try to become a full-time author.  I do 
have a whole gaggle of machines downstairs in my basement, but I have 
not yet had the time to get them all installed and configured, etc....

	I can say that I set up one of the first central log-processing 
servers when I was working at AOL, and we would have killed for tools 
like lire.  I did much the same when I was working at Skynet, the 
largest ISP in Belgium.

	What I would recommend is the same sort of thing.  Set up a 
central log processing system, and move the logs from all the various 
systems on the network to that machine, where they can be processed 
by lire or other tools.  That central log processing machine could 
also be part of an operational, real-time, network monitoring and 
management system, using tools like nagios, rrdtool, Net-SNMP, etc....

>>  >  4.  Rigid storage structure - The database structure where the log
>>  > data  is stored forces logs of all different types to conform to a
>>  > single  table definition.
>>
>>  	Well, lire has this same problem, to a degree.
>
>  I can believe it, but isn't the DFL concept a step/leap in the right
>  direction?

	Yes.  Note that lire is moving away from using a common central 
log format, and into a database model themselves.  However, I believe 
that they are going about this in an intelligent manner, and this 
will be a further improvement over what they have today.

>  This last issue I added mostly for completeness--It is one that most
>  logging systems have.  Most logging systems tack on an
>  alerting/notification mechanism, but the Lire manuals are very explicit
>  on this topic:  "Lire is a batch processor, it isn't a real-time log
>  analyzer" (p. 5).

	Yup.  Lire doesn't do real-time.

>  It is my contention that log analysis and alerting cannot, as of now, be
>  completely automated. Good analysis is the result of good
>  interpretation, which still requires human intervention.  For the
>  forseeable future, someone will have to read and interpret the logs.

	Agreed.

>  I have looked at most of these, paying particular interest to MRTG,
>  RRDTool.  I completely agree with you.

	IMO, mrtg is great for monitoring network devices, but not so 
great for monitoring general-purpose systems.  This is why Tobias 
created rrdtool, which started out in life as mrtg 2.

	However, rrdtool is still only part of the equation, since that 
primarily covers trending.  You still need something like nagios as 
well.

-- 
Brad Knowles, <brad.knowles at skynet.be>

"They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety."
     -Benjamin Franklin, Historical Review of Pennsylvania.

GCS/IT d+(-) s:+(++)>: a C++(+++)$ UMBSHI++++$ P+>++ L+ !E-(---) W+++(--) N+
!w--- O- M++ V PS++(+++) PE- Y+(++) PGP>+++ t+(+++) 5++(+++) X++(+++) R+(+++)
tv+(+++) b+(++++) DI+(++++) D+(++) G+(++++) e++>++++ h--- r---(+++)* z(+++)

-- 
To UNSUBSCRIBE, email to questions-request at logreport.org with a subject of 
"unsubscribe". Trouble? Send an email with subject "help" to 
questions-request at logreport.org