[LogReport Development] Bug: Integer Overflow causesendless loop

[Skora, Thomas]

Hi!

Yes, the overflow problem seems to be caused in DBD::SQLite2 in vdbe.c
where signed int's are used in the custom function dispatchers. But I
wonder why it works with minimal examples on the same box where Lire is
running, maybe constant folding and string conversion avoids the integer
arithmetic in such examples. I've tried to exchange all relevant int
declarations by "long long" in DBD::SQLite2 but that never worked as
expected and till now I have not the time to fix it really. It's
reported to the maintainers of the module.

I've also tried to use DBD::SQLite instead of DBD::SQLite2 in Lire since
it seems to use 64 bit arithmetic by default, but it also would cause
additional migration effort and never worked directly as fast
workaround.

Regards,
Thomas

> -----Original Message-----
> From: development-bounces at lists.logreport.org 
> [mailto:development-bounces at lists.logreport.org] On Behalf Of 
> Joost van Baal
> Sent: Sunday, July 29, 2007 11:26 PM
> To: Skora, Thomas
> Cc: LogReport Development List
> Subject: Re: [LogReport Development] Bug: Integer Overflow 
> causesendless loop
> 
> Hi,
> 
> Op Tue 24 Jul 2007 om 06:57:07 +0200 schreef Skora, Thomas:
> > 
> > I'm have a problem with Lire 2.0.1 (also with 2.0.2 - 
> mainly the prior
> > release is used since the distribution supports it) when 
> big transfers
> > are logged in a squid log file and the subreport "Requests 
> by Size" is
> > generated. The command I use is:
> > lr_log2report --output-format xml squid_access squid.log report.xml
> > 
> > When a line with a file transfer bigger than 2^31 bytes is 
> processed by
> > lire it hangs in the while loop from line 397 (sub create_entry) in
> > Rangegroup.pm because the variable $value is filled with an negative
> > value. The error seems to appear somewhere while the function
> > Lire::SQLExt::LrRangegroup::lr_rangegroup_geo returns its 
> big (>2^32) to
> > SQLite because the rangegroup column returned by SQLite 
> after the query
> > contains a negative value.
> > 
> > I thought that was an DBD::SQLite2 issue, but on the test 
> box (64 bit)
> > example programs simulating such a constellation work fine.
> > 
> > The attached patch moves the problem to some gigabytes 
> above the actual
> > limit by adding a constant to negative values. It not 
> really fixes the
> > problem but omits the endless loop for several million log 
> lines. The
> > patch contains also a second fix for 
> Lire::SQLExt::LrRangegroup where
> > bad formatted decimal numbers (, instead of . as decimal comma) from
> > SQLite cause many perl warnings.
> 
> Thanks a lot for this patch, it is very much appreciated!
> 
> I'll take a closer look at it soonish, will try to test it 
> and will try
> to get it shipped with next Lire release.
> 
> However, isn't the overflow problem caused by a bug in SQLite (or
> DBD::SQLite2)?  I don't think the Lire code is what should be fixed
> here.  Did you try to locate the bug?  Did you report it to the SQLite
> (or DBD::SQLite2) maintainers?
> 
> (the patch, inline this time:
> 
> diff -ur /root/Lire/Rangegroup.pm Lire/Rangegroup.pm
> --- /root/Lire/Rangegroup.pm    2006-08-29 10:25:06.000000000 +0200
> +++ Lire/Rangegroup.pm  2007-07-24 17:54:16.000000000 +0200
> @@ -396,7 +396,7 @@
>      my $start = $self->_param_value( 'range_start');
>      my $scale = $self->_param_value( 'size_scale' );
>  
> -
> +    $value = -$value + 2**32 if ($value < 0); # workaround 
> for overflow problem
>      while ( $value < $start || $value >= ( $start + $length ) ) {
>          $start += $length;
>          $length *= $scale;
> diff -ur /root/Lire/SQLExt/LrRangegroup.pm Lire/SQLExt/LrRangegroup.pm
> --- /root/Lire/SQLExt/LrRangegroup.pm   2005-04-26 05:15:49.000000000
> +0200
> +++ Lire/SQLExt/LrRangegroup.pm 2007-07-24 18:01:30.000000000 +0200
> @@ -24,6 +24,7 @@
>      $v = $min if defined $min && $v < $min;
>      $v = $max if defined $max && $v > $max;
>  
> +    $v =~ s/,/./;              # fix: numbers are sometimes 
> in bad format 
>      return undef if $v < $start;
>  
>      my $range_start = $start;
> 
> 
> 
> )
> 
> Thanks!
> 
> Bye,
> 
> Joost
> 
> 
> -- 
> .    .                           Log Analysis and Report Generation
> | '.|        /^LogReport$/
> | Lire                                    http://www.logreport.org/
>