[LogReport Development] Bug: Integer Overflow causesendless loop

[Joost van Baal]

Hi again Thomas,

Op Mon 30 Jul 2007 om 10:20:31 +0200 schreef Skora, Thomas:
> 
> Yes, the overflow problem seems to be caused in DBD::SQLite2 in vdbe.c
> where signed int's are used in the custom function dispatchers. But I
> wonder why it works with minimal examples on the same box where Lire is
> running, maybe constant folding and string conversion avoids the integer
> arithmetic in such examples. I've tried to exchange all relevant int
> declarations by "long long" in DBD::SQLite2 but that never worked as
> expected and till now I have not the time to fix it really. It's
> reported to the maintainers of the module.

At http://rt.cpan.org/Public/Bug/Display.html?id=28448, I see, in july
2007.  Thanks for that!  I feel the bug could best get fixed in
DBD::SQLite2 (and not in Lire).  However, it seems we'd have to wait for
quite some more time (I've just checked the status at
http://rt.cpan.org/Public/Dist/Display.html?Name=DBD-SQLite2 .)

> I've also tried to use DBD::SQLite instead of DBD::SQLite2 in Lire since
> it seems to use 64 bit arithmetic by default, but it also would cause
> additional migration effort and never worked directly as fast
> workaround.

Perhaps we should try DBD::SQLite3?  Would Lire code need to get changed
drastically for that?

Or perhaps a note could get added to the Lire documentation.  I am
really hesitating to clutter the code with workarounds for bugs in other
software.

Thanks, Bye,

Joost


> > Op Tue 24 Jul 2007 om 06:57:07 +0200 schreef Skora, Thomas:
> > > 
> > > I'm have a problem with Lire 2.0.1 (also with 2.0.2 - 
> > > mainly the prior
> > > release is used since the distribution supports it) when 
> > > big transfers
> > > are logged in a squid log file and the subreport "Requests 
> > > by Size" is
> > > generated. The command I use is:
> > > lr_log2report --output-format xml squid_access squid.log report.xml
> > > 
> > > When a line with a file transfer bigger than 2^31 bytes is 
> > > processed by
> > > lire it hangs in the while loop from line 397 (sub create_entry) in
> > > Rangegroup.pm because the variable $value is filled with an negative
> > > value. The error seems to appear somewhere while the function
> > > Lire::SQLExt::LrRangegroup::lr_rangegroup_geo returns its 
> > > big (>2^32) to
> > > SQLite because the rangegroup column returned by SQLite 
> > > after the query
> > > contains a negative value.
> > > 
> > > I thought that was an DBD::SQLite2 issue, but on the test 
> > > box (64 bit)
> > > example programs simulating such a constellation work fine.
> > > 
> > > The attached patch moves the problem to some gigabytes 
> > > above the actual
> > > limit by adding a constant to negative values. It not 
> > > really fixes the
> > > problem but omits the endless loop for several million log 
> > > lines. The
> > > patch contains also a second fix for 
> > > Lire::SQLExt::LrRangegroup where
> > > bad formatted decimal numbers (, instead of . as decimal comma) from
> > > SQLite cause many perl warnings.



-- 
.    .                           Log Analysis and Report Generation
| '.|        /^LogReport$/
| Lire                                    http://www.logreport.org/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 827 bytes
Desc: Digital signature
Url : http://lists.logreport.org/pipermail/development/attachments/20080209/d5ea8258/attachment.bin