Downtime due to Apache AH00060

To start the weekend off with a bang my Apache webserver failed to revive after the log rotation service had issued a restart. I’m hosting this website on a Raspberry Pi 3 so my first concern is always memory card corruption and data loss. Thankfully those fears turned out to be unfounded, but what actually went down?
From the httpd error_log:

AH00494: SIGHUP received.  Attempting to restart
AH00060: seg fault or similar nasty error detected in the parent process
...
# After a forced restart of httpd
AH00098: pid file /var/run/httpd.pid overwritten -- Unclean shutdown of previous Apache run?

As previously mentioned, the issue occurred as logrotate was issuing a restart. With Slackware Linux, the default logrotate configuration for the httpd server consists of the following statements:

# /etc/logrotate.d/httpd 
/var/log/httpd/*_log {
  ...
  postrotate
    /etc/rc.d/rc.httpd restart
  endscript
}

The “restart” argument from the “postrotate” section will simply invoke the apachectl (Apache HTTP Server Control Interface) script which in turn verifies the configuration and restarts the Apache httpd daemon:

# /etc/rc.d/rc.httpd
case "$1" in
...
'restart')
    /usr/sbin/apachectl -k restart

Usually this works as expected but once in a blue moon it doesn’t. I gave it a go but I could not identify the issue with my current httpd log level. I don’t really need more “noise” in my logs, so I’ll leave it at that for now.

I might consider changing the httpd logrotate configuration from passing “restart” to “force-restart” as it seems the developers are already familiar with the apachectl (Slackware?) issue:

# /etc/rc.d/rc.httpd
case "$1" in
...
'force-restart')
    # Sometimes restarting through apachectl doesn't do the trick.
    /usr/sbin/apachectl -k stop
    killall httpd
    # Remove both old and new .pid locations:
    rm -f /var/run/httpd.pid /var/run/httpd/httpd.pid
    /usr/sbin/apachectl -k start

For the record, this was the first time I experienced this error after 900 days of running Slackware ARM.

Roger hosts his website off of a Raspberry Pi!

This post, re-published here with permission, was originally published on Roger’s site here.