how to dynamically set report configuration variables (was: Re: how to add a superservice ...)

[Joost van Baal]

Hi Hengel,

On Thu, Oct 17, 2002 at 05:54:01PM -0400, Hengel Zambrano wrote:
> On Thu, 17 Oct 2002 13:58:59 +0200
> Joost van Baal <joostvb at logreport.org> wrote:
> > On Wed, Oct 16, 2002 at 06:15:56PM -0400, Hengel Zambrano wrote:
> > > Right now I have the following doubt: How can I indicate that the
> > > period of the logs my service should take. I mean, if I run:
> > > "lr_log2report serversybase < mylog" the report contains
> > > information about ALL the log entries. I would like to filter using
> > > the date, for instance from 2002/11/01 to 2002/11/02. It is because I
> > > have planed to put the logreport at the crontab to
> > > get daily information about my logs.
> > 
> > Currently, there are different ways to do this, all with their pros and
> > cons.  The easiest one might be just to change your logrotating setup.
> > Another one is to convert your complete logs to dlf, and split the dlf
> > in chunks which should result in one report each.  A simple awk or perl
> > script could take care of this: the dlf has timestamps in
> > seconds-since-epoch format at fixed fields.  A third way to handle it is
> > to use a filter-spec element in your report configuration file.  You can
> > use the | syntax.  E.g. take a look at www/filters/select-url.xml .
> > This is used in www.cfg.  There is a `gt' and a `lt' element you can use
> > in a <lire:filter-spec>, these are useful when tackling your problem.
> > 
> > Yet another solution would be to wrap all subreports in a per-day
> > timegroup: e.g. change
> > 
> >  top error-causing query types, by nof occerence
> >   foo           12
> >   bar           10
> >   blah           8
> > 
> > into
> > 
> >  top error-causing query types, by nof occerence, by day
> >   2002-10-11      10
> >    foo             6
> >    bar             4
> >   2002-10-12      12
> >    blah            8
> >    foo             4
> > 
> > You'd get a cumulative report, when processing the current sybase log on
> > a daily basis from cron.  You could take a look at e.g.
> > email/reports/deliveries-by-period-by-status.xml for an example of a
> > somewhat similar report.

> I followed your recomendation about use an cumulative report as it:
> 
>    2002-05-06 ................................................ 9
> 	a	1
> 	b	2
> 	c	3
>    2002-05-07 ................................................ 0
>    2002-05-08 ................................................ 0
>    2002-05-09 .............................................. 188
> 	xxx	1
> 	yyy	2
> 	zzz	3
> 

> Now, I would like to know how can I use a "dynamic" parameter (like
> the ones on the cfg file). I want to use the "system date" like a
> parameter in order to filter just the logs for current date, I know I
> can filter it in
> the log2dlf sybase plugin, but I want to use like a parameter because
> in that way I can select another dates. 

> Is there any way to pass this dinamycs parameters from the command
> line to log2report or log2mail ??? It maybe solve my issue.

lr_log2report(1) and lr_log2mail(1) are described in their manpages, they offer
no option for passing a variable as used in a reports' configuration file.

Both commands have a `-c report_cfg' option.  What you could do is: create a
customized one-time report configuration file which contains the actual date
variable, just before running lr_log2report.

However, all this doesn't look really attractive.  I guess using a
yet-to-be-written date-splitter would be better.  E.g. something like this:

 lr_split_date --chunk=daily --timestamp=3 < log

lr_split_date splits stdin into a number of files, named after the timerange
they report on.  The files are created in the current working directory.

--chunk  one of `daily' (files are named e.g. 2002-10-25), `weekly' (files are
         named e.g. 2002-W13), `monthly' (files are named e.g. 2002-03), etc.
--timestamp  either `syslog': a timestamp used by syslog created files, lines
         look like e.g. `Oct 25 21:01:36', and are found at the start of each
         line; or e.g. `7' (or any other digit): a timestamp is expected at
         field 7 of each line, in seconds since unix epoch format.

Such a filter would be very helpful, no changes to any Lire script would be
needed, and your problem would be solved (I guess).

(Yet _another_ way to tackle it would be to allow something like

 requests-by-period          period=$LR_WWW_REPORT_PERIOD

instead of

 requests-by-period          period=1d

in e.g. the www.cfg report configuration file.  Parameter assignment statements
would be subject to shellexpansion.  I believe this would need a change to the
&Lire::ReportConfig::parse_param_line() code.  I don't know if this would be a
sane thing to do, perhaps Francis Lacoste can share some light on it.

It would allow something like:

  export LR_SYBASE_REPORT_DATE=`date +%Y-%m-%d`; lr_log2report sybase < ... > ...

while

 some-report-name date=$LR_SYBASE_REPORT_DATE

should be in your-sybase-superservice.cfg.)

Bye,

Joost

-- 
.    .                                        http://logreport.com/
| '.|                        /^LogReport$/
| Lire                                        http://logreport.org/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://lists.logreport.org/pipermail/development/attachments/20021025/91009ec5/attachment.bin