elegant solution for nuking httpd logs

rambetter

rambetter

Code: 
nlandys@daffy# ls -alS /var/log | head
total 61420
-rw-r--r--   1 root  wheel    60976240 Dec 16 00:47 httpd-access.log
-rw-r--r--   1 root  wheel      988944 Dec 16 00:41 httpd-error.log
-rw-r--r--   1 root  wheel       96571 Dec 16 00:53 messages
-rw-------   1 root  wheel       81286 Dec 16 00:50 cron
-rw-------   1 root  wheel       80737 Dec 16 00:54 auth.log
-rw-r--r--   1 root  wheel       60544 Nov  1 00:20 wtmp.1
-rw-r--r--   1 root  wheel       55528 Sep 30 23:40 wtmp.2
-rw-r--r--   1 root  wheel       47432 Sep  1 01:27 wtmp.3
-rw-r--r--   1 root  wheel       46552 Nov 30 20:58 wtmp.0

What is an elegant way to not let the httpd logs get too large? These logs contain all hits since my server was started 6 months ago. Is there a preferred automated way to rotate logs and remove old ones?

Code: 
nlandys@daffy# pkg_info -Qao | grep apache
apache-2.2.9_5:www/apache22
apache-ant-1.7.0_2:devel/apache-ant
nlandys@daffy# uname -a
FreeBSD daffy.nerius.com 7.0-RELEASE FreeBSD 7.0-RELEASE #0: Fri Feb 29 18:56:41 PST 2008     root@daffy.nerius.com:/usr/obj/usr/src/sys/DAFFY  i386
 
Use /etc/newsyslog.conf mate.

Example entries:
Code: 
/var/log/httpd-access.log               600  9     100  *     JN
/var/log/httpd-error.log                600  9     100  *     JN

More in man newsyslog.conf
 
And a third option is logrotate that exists in the ports collection. I find it more flexible that piped logs for apache, as it can deal with any type of logs from any service. The drawback it you have a little more to administrate with a "third" deamon.
 
@vermaden:

This is probably due to what your used to. I(and my accomplices) have a central syslog server where all critical logs are sent and do not want this tainted with httpd logs, also this is a mixed environment long beyond freebsd. httpd logs are part of the non-critical system(httpd access logs especially), and on our virtual user host every user has his/hers http logs(currently about 600).

As a secondary log rotate mechanism (independent of httpd and ability of the software creating the logs) logrotate is quite a nice piece of software that does it's job. It stops the service rotate the logs and start the service again(extremely simplified though), and it's contained to a single host.

Also logrotate isn't a deamon, it's run by cron/periodic on a hourly basis.
 
Thanks for the input guys. I have decided to use newsyslog. So far I have these lines in my /etc/newsyslog.conf file:

Code: 
/var/log/httpd-access.log               644  7     100  *     JN
/var/log/httpd-error.log                644  7     100  *     JN

The log is rotated and compressed just fine, however after the rotation the logs don't get written to by the httpd process (/usr/ports/www/apache22 apache-2.2.11). Both new log files are created, but they don't contain information about any further hits to the web server. After i /use/local/etc/rc.d/apache22 restart the process, the logs are written to just fine. So how would I fix this? I read the newsyslog.conf man page, and saw a path_to_pid_file field, and I saw the signal_number field. Should I change the flags from "JN" to "J" or "JC" and add a pid file and signal number to newsyslog.conf? Which signal number? Or is there another way to do this?

What it boils down to is this. What lines should I add to newsyslog.conf to get log rotation for apache22?
 
OK, so I am going to try these exact lines in my newsyslog.conf:

Code: 
/var/log/httpd-access.log               644  10    1000  *     J    /var/run/httpd.pid
/var/log/httpd-error.log                644  10    1000  *     J    /var/run/httpd.pid

What this will do is when rotation is needed (when log files get over 1000 Kbytes), it will do the rotation and then send a SIGHUP to the process id stored in /var/run/httpd.pid. Now I'm not sure how a SIGHUP affects the httpd process. Is this what I want to do? If a user is in the middle of a large download will a SIGHUP cause the user's download to break? Will the SIGHUP correctly cause the httpd process to start writing to the newly created log files?
 
Is there any program for rotating log files, but also, normal users can have access to it?

Otherwise, a custom shell script run by cron seems to be the only viable solution?
 
@rambetter:

I do think apache waits until the daemon has no more active connections, before it recycles the pid. This however in a prefork enviroment. This is where logrotate is an advantage, as it rotates the logs and then call apachectl restart to recycle the PID's

@bsddaemon:

If you have logrotate installed on your system, you can add an entry like this to your personal crontab:
Code: 
1 3 * * * user /usr/local/bin/logrotate /home/user/mylogrotate.conf

In mylogrotate.conf you can then add and remove any logrotation you want(that you own), with any preferences that you wish to use. Or if you have access to the periodic catalogues just pop a simple shell script in it's daily catalog.
 
I found out that the signal I want to send to the httpd parent process is SIGUSR1, or 30 on FreeBSD. See here: http://httpd.apache.org/docs/2.2/stopping.html under "Graceful Restart". SIGHUP to httpd was causing an active download to abort prematurely. So I am now going to try the following lines in my newsyslog.conf:

Code: 
/var/log/httpd-access.log               644  10    1000    *     J    /var/run/httpd.pid    30
/var/log/httpd-error.log                644  10    1000    *     J    /var/run/httpd.pid    30

The one thing I can see that is wrong is that there will be an interval of time between rotating/creating the new log file(s) and sending the signal 30 to httpd. During that time, will HTTP requests not be logged to the logfiles? Will potentially several log events be lost?

[LATER}

OK I have now tested this configuration and all is working OK. I was in the middle of a huge download right as the newsyslog cron job triggered, and I verified that a new log file was created during my download without interrupting the download. I did some other web hits during this huge download, and those webhits were logged just fine in the log file. So all works now!!
 
You must log in or register to reply here.
Back
Top