doing syslog-like things on windows

[Joost van Baal]

Hi Jim,

First of all: can I quote you on a public forum?  :)

On Thu, Mar 04, 2004 at 05:47:55PM -0600, Jim Lancaster wrote:
> Not a problem. With all of my pesky questions, I'm sure I'm working your
> last nerve.

Nah, we like some traffic on our lists; thanks for your posts!

> However, if you don't mind, I'd like to share with you what
> I'm doing.  Perhaps, after reading this, you could offer your opinion.
> 
> I have focused my development (perl,apache,mysql,windows) over the last
> several months on gathering and storing log data. (I boldly assumed I
> would eventually get to analysis and reporting.)  I've already built a
> couple of working scripts (agents) that can read and process Windows
> event logs and Symantec Anti-Virus logs. The way they work is this:
> 
> The agent reads an XML-formatted configuration file to obtain the last
> record read from the log file during the last poll, then 'attaches' to
> the log file, and hands it off for processing.  The main body of the
> agent reads each event, parses it into XML, fills in missing fields or
> discards ill-formed events as necessary (validates), and [optionally]
> filters out uninteresting records.   The results are handed off to an
> output procedure which forwards them on to the central server and/or
> [optionally] writes the results to a log file.  (I use the latter mostly
> for testing.) Finally, the configuration file gets updated with the last
> record read.
> 
> In addition to processing events, the agent generates a 'heartbeat' to
> let the server know that the agent is still active.  A log file may not
> contain any new events since the last poll, so it would be useful to
> know that the agent is working nevertheless.
> 
> The XML configuration file provides an opportunity to add additional
> data (db tablename(or DLF!),customer number, location, etc.) to each XML
> event record. It can also be used to configure 'ignore' criteria, if so
> desired.  IMHO, the overriding consideration should always be on
> isolating the truly useful from the merely interesting.  There is no
> need to clog up the works with 'merely interesting' events.  The
> (optional) 'ignore' filters give me the opportunity to ditch huge chunks
> of data before it ever reaches the wire.

To me it seems like you've reimplemented Unix syslog plus some extra
stuff for Windows.  But I'm not a Windows expert, and hardly qualify as
a syslog expert.  I'm pretty sure the people on
loganalysis at lists.shmoo.com (see
http://lists.shmoo.com/mailman/listinfo/loganalysis ) can say much more
sane things about what you've done.

> All of this I have working.  The next step is where I'm headed unless
> someone stops me from wasting my time. <g>
> 
> My itention is to use SOAP (via SOAP::Lite?) to transmit the XML event
> data to the server over HTTPS. The listening script will simply parse
> the incoming XML into records and insert them into the appropriate
> tables in the database.  Initially, I was going to create a different
> table for every log type, but your brilliant DLF concept will prove much
> better.
> 
> The XML event record contains all of the fields in a standard DLF
> schema, plus the tablename.  It can also contain optional, user defined
> fields like customer number, location, or anything else that an MSP
> might use.  The tablename tells the XML parser where to put the record.
> 
> Well, that's it.
> 
> What do you think?

Well, it's pretty cool the DLF setup gave you inspiration.  About the
reporting setup, Francis Lacoste might more likely be able to say
something sane than I am.

Bye,

Joost

-- 
.    .                                        http://logreport.com/
| '.|                        /^LogReport$/
| Lire                                        http://logreport.org/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://lists.logreport.org/pipermail/questions/attachments/20040305/249be689/attachment.bin