syncing problem on Solaris and *BSD systems, 0 lines in dlf (was: Re: sendmail syslog files ?)

[Faruk Celik]

Hi Joost,

I changed the  ARCHUVE to ARCHIVE in
$(prefix)/etc/lire/defaults.local

(you know... U is close to I on the keyboard :] )

I finally achieved to get a report by following 
the steps that you sent. Thank you.

I hope the bug that you find is not a "tanker bug" in
the movie Starship Troopers :)

Bye,
Faruk

--- Joost van Baal <joostvb at logreport.org> wrote:
> On Wed, Nov 07, 2001 at 04:55:13AM -0800, Faruk
> Celik wrote:
> > 
> > --- Joost van Baal <joostvb at logreport.org> wrote:
> > > On Tue, Nov 06, 2001 at 10:54:46PM -0800, Faruk
> > > Celik wrote:
> > > > 
> > > > I'm trying to analyze our Sendmail SMTP logs.
> > > > I'm using Sendmail on Solaris 2.7 and 
> > > > lire-full-20011017.tar.gz.
> <snip>
> > > > and LIRE created an empty /tmp/report.txt and
> <snip>
> > "0 lines of information from the 37458 lines in
> the
> > log......"
> <snip>
> > >  grep 'sendmail\[' /var/log/syslog | lr_run
> > > lr_log2report /tmp/error email sendmail \
> > >    > /tmp/report.txt
> <snip>
> > 
> > After I've read your mail I've changed my
> > $(prefix)/etc/lire/defaults.local like that:
> > 
> > -------------My new defaults.local
> STARTED-----------
> > # $(prefix)/etc/lire/defaults.local - local
> settings
> > for Lire 
> > #   system on mymachine ;)
> > # to be sourced by $(prefix)/etc/lire/defaults
> > # automatically generated by lr_config(1) on Tue
> Nov 
> > 6 16:48:45 EET 2001
> > # --------------------------------
> > # I take out the my machine related informations. 
> > # --------------------------------
> > 
> > KEEP=1
> > DEBUG=1
> > ARCHUVE=1
> 
> Oops, that should be 'ARCHIVE'
>                           ^
> 
> > LOGGING=stderr
> > DEFAULT_OUTPUT_FORMAT=txt
> > LR_TARGET_USER=sysadmin
> > LR_USERLEVEL=advanced
> > LR_MAX_MEMORY=40Megs
> > jobfiles_weekly=""
> > jobfiles_daily=""
> > 
> > -------------My new defaults.local
> FINISHED-----------
> > 
> > And I got a lot of debug_output to console. I
> attached
> > a file named "lire_shell_output.txt" to this mail.
> 
> Thanks!  This helps.
> 
> <snip>
> > I'm not going to give up Joost ;-]. 
> > Because I liked your project. 
> 
> Well, thanks!
> 
> > And I'm sure you know the absence of a
> > real/nice/detailed "sendmail log analyser" tool. 
> > 
> > (I checked Sendmail FAQ
> > http://www.sendmail.org/faq/section4.html#4.7 .
> > Most of them didn't give me number of rejected ,
> > successfull mails and they're producing so simple
> > reports.)
> <snip>
> 
> Just sent the maintainer an email, thanks for the
> pointer.
> 
> 
> 
> OK, the debug output features:
> 
> email sendmail lr_tag-20011107134547-27784
> lr_log2xml info gonna run 
>   sendmail2dlf  > 
>  
>
/$(prefix)/tmp/lr_log2xml.sendmail.lr_tag-20011107134547-27784.27797.dlf
> 
> email all lr_tag-20011107134547-27784 sendmail2dlf
> info read 39194 lines, 
>   output 62197 lines
> 
> email sendmail lr_tag-20011107134547-27784
> lr_log2xml info lr_db_store 
>   lr_tag-20011107134547-27784 dlflines        0
> succeeded
> 
> email all lr_tag-20011107134547-27784 lr_dlf2xml
> info 'DLF contains 0 
>   records; start and end time are unavailable
> 
> I guess we've hit a bug.
> 
> The problem seems to be the generated dlf file,
>
(/$(prefix)/tmp/lr_log2xml.sendmail.lr_tag-20011107134547-27784.27797.dlf
> in your case ), is not synced to disk, before it's
> being opened again,
> to run
> 
>  LINES=`wc -l < $DLFFILE`
> 
> (that's in the lr_dlf2xml code.)
> 
> When I tested the script, I had to add some sync's. 
>  I made a mistake,
> however.  The sync call is not portable.
> 
> SunOS 5.7 fsync(3C)
> 
>      The fsync()  function  is  different  from 
> sync(),
>      which  schedules  disk I/O for all files  but
> returns before
>      the I/O completes. The fsync() function forces
> all outstand-
>      ing  data  operations to synchronized file
> integrity comple-
>      tion
> 
> The GNU file utilities `sync' "ensures everything in
> memory is written
> to disk."
> 
> The linux sync systemcall:
> 
>        According to  the  standard  specification 
> (e.g.,  SVID),
>        sync()  schedules  the  writes,  but may
> return before the
>        actual writing is done.   However,  since 
> version  1.3.20
>        Linux  does actually wait.
> 
> BSD-ish systems need fsync(2), like Solaris, to get
> guaranteed flush to
> disk.  Unfortunately, there's no commandline
> interface on Solaris to
> call fsync.  FreeBSD has got /usr/bin/fsync .
> OpenBSD has no commandline
> fsync.  Perl doesn't offer an fsync interface.  We
> really _need_ the
> sync, since what's done is:
> 
>  $TODLF $FLAGS > $DLFFILE
> 
>  LINES=`wc -l < $DLFFILE`
> 
>  lr_dlf2xml $SUPERSERVICE $REPORT_CFG $DLFFILE
> 
> and lr_dlf2xml really needs a file to operate on.
> 
> I don't think cludging with tee(1) will increase
> robustness.
> 
> I need some thinking on this.  I'll get back to it
> tomorrow.  It surely
> is an interesting bug :)
> 
> 
> OK, there is a cludgy workaround:
> 
> I used ${prefix} /usr/local/bin
> 
> cochon at abramowitz:~% export
>  
>
PATH=/usr/local/bin:/usr/local/libexec/lire:/usr/local/libexec/lire/email:/usr/bin:/bin
> 
> cochon at abramowitz:~% grep 'sendmail\['
> /var/log/syslog | sendmail2dlf > \
>   /tmp/sendmail.dlf
> 
> cochon at abramowitz:~% grep 'sendmail\['
> /var/log/syslog | \
>  lr_run sendmail2dlf > /tmp/sendmail.dlf
> 
> cochon at abramowitz:~% lr_run lr_dlf2xml email
> /usr/local/etc/lire/email.cfg \
>  /tmp/sendmail.dlf > /tmp/sendmail.xml
> 
> cochon at abramowitz:~% lr_run lr_xml2ascii <
> /tmp/sendmail.xml > \
>  /tmp/sendmail.report
> 
> This will give you an ascii report.
> 
> You'll hear about the sync problems.  I'd very much
> like to get this bug
> 
=== message truncated ===

> ATTACHMENT part 2 application/pgp-signature 



__________________________________________________
Do You Yahoo!?
Find a job, post your resume.
http://careers.yahoo.com

-- 
To UNSUBSCRIBE, email to questions-request at logreport.org with a subject of 
"unsubscribe". Trouble? Send an email with subject "help" to 
questions-request at logreport.org