Help - Search - Members - Calendar
Full Version: Can webalizer handle huge log files?
The Planet Forums > Control Panels > Plesk
autoquartz
My webalizer is not updating some days of the months (the ending couple days) on a vhost. This vhost has some traffic going on. I have experienced the same with my other server as well. I turned the quiet mode off in webalizer.conf and ran it manually, this is what I get. Seems like it is ignoring all the records. Any clue?

Webalizer V2.01-10 (Linux 2.4.21-15.0.3.ELsmp) English
Using logfile /var/log/httpd/access_log (clf)
DNS Lookup (10): 82 addresses in 8.57 seconds, 9/sec
Using DNS cache file /var/lib/webalizer/dns_cache.db
Creating output in /var/www/usage
Hostname for reports is 'enzo.justhostit.net'
Reading history file... /var/lib/webalizer/webalizer.hist
Reading previous run data.. /var/lib/webalizer/webalizer.current
26387 records (26387 ignored) in 0.11 seconds
DeadEye686
I had this same problem, so I did some searching around the Plesk forum, and it turns out that the Plesk statistics program has a problem with log files over 2GB in size (Webalizer by itself can handle them, but the statistics program can't pipe them to it). I just set up log rotation to create a new log file at 500 MB.
bsykes
On Linux systems, you'll generally run into some strange problems after a file reaches >= 2GB.
autoquartz
The access_log for that vhost is only at 202MB icon_sad.gif
Cyan Smoke
autoquartz, the text you pasted here seems OK...just that you webalizer processed that data.

Kevin, the main issue with files > 2 GB is with log_rotate...webalizer works like a charm.
autoquartz
Actually I just noticed that this is not just for one vhost. All the vhosts' statistics stopped updating on August 29. Anyone has any idea? icon_question.gif
DeadEye686
I'm sure you checked this, but the statistics cronjob is still there, right?
autoquartz
It is still showing when I do a ls /etc/cron.daily
autoquartz
It is still showing nothing from August 30 onwards after I ran this.

[root@enzo cron.daily]# ./00webalizer
Webalizer V2.01-10 (Linux 2.4.21-15.0.3.ELsmp) English
Using logfile /var/log/httpd/access_log (clf)
DNS Lookup (10): 145 addresses in 12.83 seconds, 11/sec
Using DNS cache file /var/lib/webalizer/dns_cache.db
Creating output in /var/www/usage
Hostname for reports is 'enzo.justhostit.net'
Reading history file... /var/lib/webalizer/webalizer.hist
Reading previous run data.. /var/lib/webalizer/webalizer.current
Saving current run data... [09/04/2004 12:49:29]
Generating report for September 2004
Generating summary report
Saving history information...
52384 records (50997 ignored) in 0.28 seconds


Strange thing is the 9/2004 seems to be there...

[root@enzo cron.daily]# more /var/lib/webalizer/webalizer.hist
4 2004 1019 793 72 1151 24 30 796 62
5 2004 20057 13349 158 68201 1 31 12727 193
6 2004 72619 31584 399 184354 1 30 33808 795
7 2004 121217 54809 568 622213 1 31 52870 1288
8 2004 198509 76837 723 779203 1 31 66521 1980
9 2004 37969 12018 121 167572 1 4 11771 273
autoquartz
Now it is updating again. Anyone got similar experience? I have seen this before in my other server.
This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.
Invision Power Board © 2001-2009 Invision Power Services, Inc.