Reusable Filters Proposition
Joost van Baal
joostvb at logreport.org
Tue Jan 15 16:33:22 CET 2002
Hi,
Sorry for my rather late reply... (Not too late though, as I learned
from #logreport.)
On Sun, Dec 23, 2001 at 05:58:56PM -0500, Francis J. Lacoste wrote:
>
> Reusable Filters Proposition
> ============================
>
> Here is a proposition to improve the filter interface which is
> presently embodied in the 'filter-spec' element in the
> report-specification.
>
<snip>
>
> The way to hook up those filters with the report specifications would
> be through the superservice report's configuration file (ftp.cfg,
> email.cfg, etc.) Filters would be setup by adding '|<filter_id>' lines
> along with their parameter's value. The filter would apply to _all_
> subsequent reports. For example, the dns.cfg could now be simplified
> to :
>
> top-requesting-hosts hosts_to_show=10
> top-requested-names names_to_show=10
> requesttype-distribution
> requests-by-period period=1d
>
> |filter_eq method="recurs'
> top-requesting-hosts hosts_to_show=10
> top-requested-names names_to_show=10
> requesttype-distribution
> requests-by-period period=1d
>
> |filter_eq method="nonrec"
> top-requesting-hosts hosts_to_show=10
> top-requested-names names_to_show=10
> requesttype-distribution
> requests-by-period period=1d
>
> Adjacent filter specifications would be cumulative:
>
> |filter_ne client_host="localhost"
> |filter_not_re referer="^(http://www.logreport.org|-$"
> top-referers-by-page referer_to_show=5 page_to_show=10
OK, lets see if I fully understood you. dns.cfg now is something like:
top-requesting-hosts hosts_to_show=10
top-requesting-hosts-by-method hosts_to_show=10 method='recurs'
top-requesting-hosts-by-method hosts_to_show=10 method='nonrec'
top-requested-names names_to_show=10
top-requested-names-by-method names_to_show=10 method='recurs'
Clearly, your proposal leads to a much more maintainable system, since
we can get rid of report definitions like
dns/reports/top-requesting-hosts-by-method.xml , and keep only
dns/reports/top-requesting-hosts.xml.
You store this information:
<lire:filter-spec>
<lire:eq arg1="$resolver" arg2="$method"/>
</lire:filter-spec>
in dns.cfg now ( |filter_eq method="something" ). And, indeed, we need
a place to store the description of the filter. I guess the description
can be constructed from the description of the variables in lire:field
in lire:dlf-schema. Hmm... Does lire:field allow a description?
Description would be something like:
Number of Lookups by Hosts
after filtering: Resolving method equals recursive
<snip>
> Filters would be reset whenever we encounter a new filter
> specification following a report-specification (like in the DNS
> example above). To reset the filter explicitely, we add the
> |filter_none "magic" filter.
>
> Since there are many simple filters that are possible and would be
> potentially useful I suggest to add other "magic" filters which
> wouldn't need to be explicitely defined in external files. There would
> be automatically a filter_<op> and filter_not_<op> defined for each
> schemas.
I don't really get this. Which filters do you call 'magic' and which
are not 'magic'? Is filter_eq a magic filter?
<snip>
> The Problems of the Solution
> ============================
>
> I still see some issues with this solution, I welcome all comments and
> suggestions on those (as well as on all other parts of this proposal).
>
> 1) Implicit vs explicit reset
>
> I'm not sure about the automatic reset of filters after
> report-specifications behavior that I specified above. Should we only
> reset when we encounter a |filter_none filter? Maybe this is more
> explicit, also it makes it possible to filter more records after some
> reports were generated on an already filtered input.
Currently, the order of the subreports in the .cfg file decides for the
order in the final report. Is the actual computation done in the same
order? This might lead to problems after a while... (I fear the work
needed to make this more flexible, though...)
> 2) Including informations about applied filters in the generated
> report.
>
> We have to find a way to include informations about the applied
> filters in the generated reports. For example, in the above DNS
> example, we have to include informations about the record subsets on
> which the reports were computed since they will have all the same
> title. Maybe we should generate a section header whenever a filter was
> applied? Something like : "Recursive Requests Reports". Maybe we could
> use the display-spec element in the filter specification for that
> purpose.
<snip>
See above.
On Sat, Jan 05, 2002 at 03:51:28PM -0500, Francis J. Lacoste wrote:
> On Tue, Jan 01, 2002 at 12:28:54PM +0100, Egon Willighagen wrote:
> > If we make the syntax of this configuration file more complex, isn't it
> > time to move over to XML?
>
> I thought about this. I agree that the configuration file should
> eventually be a XML file for extensibility and clarity.
>
> So until we have a good configuration infrastructure in place, I vote
> that we keep the single/simple configuration file scheme.
I even doubt wether the configuration file should be XML ever. A
configuration file is there to offer easy hooks. If people want
unlimited flexibility, they should hack in Lire itself. The current
configuration file syntax allows for enough flexibility, I guess.
Writing a userfriendly interface to the *current* configuration file is
higher on my personal wishlist then improving flexibility.
This is related to the simplicity vs exhaustivity thing: once we offer a
very exhaustive system, it will be next to impossible to write a
userfriendly configuration interface for it.
Bye,
Joost
--
Joost van Baal . . http://www.logreport.org/
. .
/^LogReport$/ . . joostvb at logreport.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://lists.logreport.org/pipermail/development/attachments/20020115/0b452686/attachment.bin
More information about the Development
mailing list