Help - Search - Members - Calendar
Full Version: HOWTO: Correct the problem with log rotation
The Planet Forums > Control Panels > cPanel/WHM
Pages: 1, 2, 3
aussie
LAST UPDATED 08-23-03

Do we have a HOWTO forum for cPanel, i guess not but whoever is in charge can move this someplace;

Since installing cpanel on some of our server, we noticed the logs in /usr/local/apache/logs/* growing out of control without ever being rotated. In /etc/logrotate.d the script called "apache" is the reason why. Its totally wrong. Its looking for files located in /var/log/httpd/ and that is not where my log files are located. In fact, we dont even have files in this directory. Its completely empty.

The first line:

/var/log/httpd/access_log /var/log/httpd/agent_log /var/log/httpd/error_log /var/log/httpd/referer_log {

should be changed to, at the very minimum;

/usr/local/apache/logs/access_log /usr/local/apache/logs/agent_log /usr/local/apache/logs/error_log /usr/local/apache/logs/referer_log {

In addition, you may want to add the follow logs to the list above
and any other files you find that needs to be rotated in this directory.

ssl_engine_log
suexec_log

You could leave out agent_log and referer_log since i dont see those as even existing in my /logs/* directory but if you have them add them.

UDDATE: 08-23-03 - Thanks dkair!!!!
httpd.pid is not located in /var/run/. On my server its located in /usr/local/apache/logs. Also in "apache" change the following line;

/bin/kill -HUP `cat /var/run/httpd.pid 2>/dev/null` 2> /dev/null || true

to

/bin/kill -HUP `cat /usr/local/apache/logs/httpd.pid 2>/dev/null` 2> /dev/null || true

End Change

logrotate.conf

In /etc/logrotate.conf where is the reference to lastlog? The default apache log includes a rotatation of lastlog but i didnt find an entry in my logrotate.conf.

My /var/log/lastlog was around 19megs so add the following to logrotate.conf so the added entry looks like this.

# no packages own lastlog or wtmp -- we'll rotate them here
/var/log/wtmp {
monthly
create 0664 root utmp
rotate 1
}


# system-specific logs may be configured here


/var/log/srv08.primenet.cc { <---- Change to the name of your cert.
weekly
rotate 1
}

/usr/local/cpanel/logs/access_log {
weekly
rotate 1

}

/usr/local/cpanel/logs/stats_log {
weekly
rotate 1

}



SSL logs:

If your running ssl on your server you will find some huge log files in /var/log if you have never rotated them. Ours was hitting 51megs. Your SSL log files will match the name of your hostname or the name of your certificate or both depending on how many SSL certs you have running. You could also add those above.

Also, do you really need to keep 4 sets of rotated logs? I rarely refer back that far unless i have some problem and i need to go back to view the logs but usually going back 1 or 2 weeks of rotated logs is sufficient. I have changed my number of backup logs in logrotate.conf to 2. Old rotated logs even going back 4 weeks could take up huge amounts of disk space if there was alot of activity. Consider compressing the rotated logs. Simply uncomment #compress which will compress your old rorated logs.

Now whats happening to xferlog? It was being rotated at one stage but no longer. The last time it was rotated was back in Nov 22 of 2002 on my box. Since then the file has grown to 25megs and not rotated. So was it lost somewhere in all the cpanel updates? One will never know since i cannot find any reference anywhere to it being rotated. If this is also the case on your box do this. If anyone knows where its being rotated let me know because i cannot find it

cd /etc/logrotate.d

is there proftpd in that directory?

FIX;

in the logrotate directory create proftpd

pico or vi proftpd

add;

Update 08-23-03 Fixed typo in the 3rd line. The pid name should be proftpd.pid, not profptd.pid. Sorry! Change it if yours is wrong.

/var/log/xferlog {
postrotate
/bin/kill -HUP `cat /var/proftpd.pid 2> /dev/null` 2> /dev/null || true
endscript
missingok
}

End Change

Save it.

That should take care of most of the rotation that Cpanel is just not doing. Cpanel 6 HELLO! can we have these fixed?

UPDATED 08-23-03 As of Cpanel7 and since i origionally posted this msg this log rotation stuff is still wrong. cPanel has not taken the time to update and fix the problems. Maybe somebody should post a TT to cPanel with a link to this thread. I dont know. What will it take to get them to fix this? My newest server, purchased 1 month ago, had to be updated as well. So if you are new to cPanel, you have some work to do. As for cPanel! rolleyes.gif
Got-Hosting
cPanel should have given licenses to RS for free.

This community is going to be the best thing that ever happened to cPanel.

Now, if I could just get my hands on one!
dkair
When making these changes to we need to rstart anything?

Also should this line not be changed to:

/bin/kill -HUP `cat /usr/local/apache/logs/httpd.pid 2>/dev/null` 2> /dev/null || true ?

The orginal had

/bin/kill -HUP ` cat /usr/run/httpd.pid 2>/dev/null` 2> /dev/null || true

There is no httpd.pid in /usr/run/
aussie
Where are you seeing that DK? No, you dont need to restart anything.
dkair
/etc/logrotate.d/apache

/bin/kill -HUP ` cat /usr/run/httpd.pid 2>/dev/null` 2> /dev/null || true

needs to be changed as well to:

/bin/kill -HUP `cat /usr/local/apache/logs/httpd.pid 2>/dev/null` 2> /dev/null || true
aussie
:eek: Yes, you are correct. Something i missed. Absolutely absurd if you ask me. I would understand a typo or mis configuration in the entire log rotation process but an outright entire rewrite of the entire process to correct paths is just INEXCUSABLE!

I have just about lost all my respect for cPanel. Just about! Im teetering on the edge. I waiting to be hacked sooner or later and when that happens because of the security issues they will find out about it.
dkair
LOL Ur funny.. I was looking for those files as well that you said were missing in the orginal paths. Then I saw the next line and did the same chk and there was no file at that location either..
aussie
QUOTE
Originally posted by dkair
LOL Ur funny.. I was looking for those files as well that you said were missing in the orginal paths. Then I saw the next line and did the same chk and there was no file at that location either..


Yep i know. I guess one should waiting till their 80gig drives shrink down to nothing and system slows to a crawl because nothing has been done about this. I dont know how many times i reported this problem and it exists on all installations. Its only worth about 10 bucks a month if that. I feel sorry for the suckers who actually have $75 external licenses and we can only thank the big HS for coming to our rescue.
freddo
/usr/local/apache/logs/access_log /usr/local/apache/logs/agent_log /usr/local/apache/logs/error_log /usr/log/apache/logs/referer_log {

should be

/usr/local/apache/logs/access_log /usr/local/apache/logs/agent_log /usr/local/apache/logs/error_log /usr/local/apache/logs/referer_log {

and since you really want to do ssl and suexec too:

/usr/local/apache/logs/access_log /usr/local/apache/logs/ssl_engine_log /usr/local/apache/logs/suexec_log /usr/local/apache/logs/agent_log /usr/local/apache/logs/error_log /usr/local/apache/logs/referer_log {

I'm just another copy and paster.

Much appreciated
Erwin
Thanks aussie. This was very helpful. icon_smile.gif The logrotate config was all wrong as you say, and my access_logs were up to 300 over in MBs!
freddo
from the look of your Avatar - you been in the States too long!
The Rowan
hmm, interesting, I got a new cpanel server and was about to this fix and noticed that i have logfiles in both locations:

/usr/local/apache/logs
cd /var/log/httpd

could it be this affects upgrades from ensim only and not new cpanel servers?


anybody having more info on this?

thanks
aussie
QUOTE
Originally posted by freddo
from the look of your Avatar - you been in the States too long!


:eek: I hope you like them!
aussie
QUOTE
Originally posted by The Rowan
hmm, interesting, I got a new cpanel server and was about to this fix and noticed that i have logfiles in both locations:

/usr/local/apache/logs
cd /var/log/httpd

could it be this affects upgrades from ensim only and not new cpanel servers?


anybody having more info on this?

thanks


Hello, these files can get huge if not kept in check and should also do the rotation on them.

/var/log/httpd? This is not a cPanel directory and its safe to remove all thes because cPanel doesnt write to these directories. You must have upgraded from Ensim and has been left over by Ensim. Just cd to /var/log and do a rm -rf httpd.

As far as for /usr/local/apache logs (SEE ABOVE) i have already specified the log rotation.

But you have other problems. You must also check /usr/local/cpanel/logs/*. These files get quite large and i suggest you add the following to the end of logrotate.cof


/usr/local/cpanel/logs/access_log {
weekly
rotate 1

}

/usr/local/cpanel/logs/stats_log {
weekly
rotate 1

}

icon_biggrin.gif
freddo
QUOTE
Originally posted by aussie
:eek: I hope you like them!
I posted that when you had the "bad driver" avatar.
jackal
QUOTE
Since installing cpanel on some of our server, we noticed the logs in /usr/local/apache/logs/* growing out of control without ever being rotated. In /etc/logrotate.d the script called "apache" is the reason why. Its totally wrong. Its looking for files located in /var/log/httpd/ and that is not where my log files are located. In fact, we dont even have files in this directory. Its completely empty.


Do not have those files.
mbarb
Hi,
There is a option in WHM tweak settings to rotate the dom logs after every stats run., This assures that the complete logs are available for awstats, webalizer or analog, to chew on, then it tosses out the dom logs. As far as I know Cpanel has never used log rotate for the dom logs
The Rowan
QUOTE
/var/log/httpd? This is not a cPanel directory and its safe to remove all thes because cPanel doesnt write to these directories. You must have upgraded from Ensim and has been left over by Ensim. Just cd to /var/log and do a rm -rf httpd.


hmmm, i just got a brand new dell xeon server witn cpanel, aree you sure that ]/var/log/httpd? does not exist in cpanel?

that would piss me of if i got an ensim upgrade to cpanel on a brandnew server.:eek:
jackal
/etc/logrotate.d/apache

is not there, think cpanel may have moved it to another path
aussie
QUOTE
Originally posted by The Rowan
hmmm, i just got a brand new dell xeon server witn cpanel, aree you sure that ]/var/log/httpd? does not exist in cpanel?

that would piss me of if i got an ensim upgrade to cpanel on a brandnew server.:eek:


Correction, if you got a new cpanel server and your seeing the directory /var/log/httpd thats ok just as long as there are no files in this directory otherwise its some kind of upgrade over the top of Ensim. If on the other hand you have upgraded over the top of Ensim the files in this this directory can be removed because its not used by cpanel. In other words /var/log/httpd/ should contain nothing!
TDD
Aussie i use pure-ftp and not prodftp

then for that what changes i should make to rotate logs ?


i guess this is onlt for prodftp:-

---------------------------
is there proftpd in that directory?

FIX;

in the logrotate directory create proftpd

pico or vi proftpd

add;

/var/log/xferlog {
postrotate
/bin/kill -HUP `cat /var/profptd.pid 2> /dev/null` 2> /dev/null || true
endscript
missingok
}
-------------------------

what will be equivalent for pure-ftp ?

Thanks
TDD
freddo
i don't believe pureftp does any logging.
Networkologist
So does this take care of the access, error, ssl and suexec files in /etc/httpd/logs?

It's a symlink, no?
Networkologist
I just received this:

error: /etc/logrotate.conf:32 duplicate log entry for /var/log/secure

logratate.conf has the following, begining with line 32:

/var/log/secure {
weekly
rotate 1
}


I don't see where it's duplicated. Should I remove it?
sqsisa
I have gotten two of these messages since applying the fix:

apache failed @ Sun May 4 03:35:32 2003. A restart was attempted automagicly.

Apache is up and running. How can I verify that this working properly and that the logs are being rotated?

Thanks,
aussie
QUOTE
Originally posted by sqsisa
I have gotten two of these messages since applying the fix:

apache failed @ Sun May  4 03:35:32 2003. A restart was attempted automagicly.

Apache is up and running. How can I verify that this working properly and that the logs are being rotated?

Thanks,


That message is not related to log rotation.
aussie
QUOTE
Originally posted by Networkologist
I just received this:

error: /etc/logrotate.conf:32 duplicate log entry for /var/log/secure

logratate.conf has the following, begining with line 32:

/var/log/secure {
weekly
rotate 1
}


I don't see where it's duplicated. Should I remove it?


Secure is already being rotated and should not be in logrotate.conf.
Networkologist
Thanks.

RS ought to compensate you. icon_wink.gif
sqsisa
Aussie, you da man! Thanks!

And those hoots ate the HOOTS icon_razz.gif
aussie
QUOTE
Originally posted by sqsisa
Aussie, you da man! Thanks!

And those hoots ate the HOOTS icon_razz.gif


:eek:
sqsisa
Damn dude! Change it back. The kids were the best part of your posts! lol

No seriously, you da man!:eek:
RisingHost
aussie,
Thanks for all the info. Maybe you should edit the original post so that it reflects all the error corrections in the replies? Just thought it might save some people some time. Thanks once again,
Risinghost
d3nnis
so which are the correct ones? aussie can update?
brichi
QUOTE
Originally posted by aussie
[BThe first line:

/var/log/httpd/access_log /var/log/httpd/agent_log /var/log/httpd/error_log /var/log/httpd/referer_log { [/B]


my apache file is more spaced out - can someone post their updated happy logrotate file?

Here is how mine looks like...

/var/log/httpd/access_log {
missingok
postrotate
/bin/kill -HUP `cat /var/run/httpd.pid 2>/dev/null` 2> /dev/null || true
endscript
}

/var/log/httpd/agent_log {
missingok
postrotate
/bin/kill -HUP `cat /var/run/httpd.pid 2>/dev/null` 2> /dev/null || true
endscript
}

/var/log/httpd/error_log {
missingok
postrotate
/bin/kill -HUP `cat /var/run/httpd.pid 2>/dev/null` 2> /dev/null || true
endscript
}

/var/log/httpd/referer_log {
missingok
postrotate
/bin/kill -HUP `cat /var/run/httpd.pid 2>/dev/null` 2> /dev/null || true
endscript
}

/var/log/httpd/ssl_request_log {
missingok
postrotate
/bin/kill -HUP `cat /var/run/httpd.pid 2>/dev/null` 2> /dev/null || true
endscript
}


--Tone
Clark
QUOTE
Originally posted by Got-Hosting
cPanel should have given licenses to RS for free.

This community is going to be the best thing that ever happened to cPanel.

Now, if I could just get my hands on one!  



THANKUTHANKUTHANKUTHANKU!!!!!!

I didn't get any real work done tonight (because I found u guys), well, not what I planned to anyways..but I am in heaven!!!!

I knew there were a million things wrong with cpanel (any my provider) but was desperate for a place to get info. You guys are the beeeesssssstttt. THANKU!!!!

I'm going to live here.

I'm going to switch to RS for sure (as soon as I find out if the network is as stable as you users are good. I can't imagine a group like you staying with a bad provider though.)

--------------
P.S. I also don't have that apache file. It's missing. Is this How To the latest? Can someone post the entire file? Thanks!
sqsisa
Clark,

You will love the forum. There is so much info here, it will take you weeks to go through. I don't have a RS server, but this forum rocks. The only thing to remember is to do a search before you post a question. You will almost always find a thread on what you want. Happy moding!
Mirek-FWH
Does any of this still apply to the current cpanel version?

Thanks,
Mirek
RisingHost
I believe so.
aussie
QUOTE
Originally posted by Mirek-FWH
Does any of this still apply to the current cpanel version?

Thanks,
Mirek


Yep, why do you think they suddenly pulled their finger out and fixed it?
Got-Hosting
QUOTE
Originally posted by aussie
Yep, why do you think they suddenly pulled their finger out and fixed it?


So, you are saying they fixed it?

Sorry for being dense.
aussie
QUOTE
Originally posted by Got-Hosting
So, you are saying they fixed it?

Sorry for being dense.


Negative, if you compare what i posted and what you have and its the same then they have not fixed it right?
Mirek-FWH
so they did fix it?
or was that some sort of oddly stated sarcastic remark?

In other words... you are not dense, I am confused about this one too, hehe

and thanks for the replies guys


edit: I checked and the paths were wrong in my current version... so I guess it still applies... doesn't seem like a hard thing for cpanel to fix..
RisingHost
aussie seems to making two conflicting msgs in his past two posts when in reality the first post is extreme sarcasm and the second one is more straight forward

Anyways what he is trying to say is that IT HAS NOT BEEN FIXED
Got-Hosting
QUOTE
Originally posted by aussie
Negative, if you compare what i posted and what you have and its the same then they have not fixed it right?


I don't have the originals to compare it to, because I applied your fix about a month ago! icon_smile.gif

Take care!
aussie
QUOTE
Originally posted by Got-Hosting
I don't have the originals to compare it to, because I applied your fix about a month ago!  :)

Take care!


The answer is NO!
Got-Hosting
QUOTE
Originally posted by aussie
The answer is NO!


Heh, yeah, I figured that out too. icon_smile.gif

Take care!
d3nnis
aussie can you post your updated complete logrotate? Kinda confuse here :confused: icon_sad.gif
webprox
QUOTE
Originally posted by d3nnis
aussie can you post your updated complete logrotate? Kinda confuse here :confused: icon_sad.gif


I'm confused aswell:confused:

Would love to try the new version of the howto icon_biggrin.gif

Thanks aussie
daveman692
What about dom logs? How can I rotate them daily?
guidster
Thanks for the effort and awareness, Aussie.
This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.
Invision Power Board © 2001-2009 Invision Power Services, Inc.