This problem seems to be related to 
restarting the server.
First, multiple HUP signals from /etc/logrotate.d/apache
may easily kill the server.
(when next -HUP is delivered when first -HUP is is still processing)

Second from time to time a SINGLE HUP signal kills the apache.
This is a command which sometime JUST KILLS apache 
when executed manually.

kill -HUP `cat /var/run/httpd.pid`

Another thing I would recommend to change in 
/etc/logrotate.d/apache
is to replace multiple entries 
by a single one
like
/var/log/httpd/access_log /var/log/httpd/error_log /var/log/httpd/agent_log
/var/log/httpd/referer_log {
   missingok
   sharedscripts
   postrotate
        /bin/kill -HUP `cat /var/run/httpd.pid 2>/dev/null` 2> /dev/null ||
true    endscript
   endscript
}

This will prevent killing apache by delivering multiple -HUP signals

Thanks!  This'll be integrated into apache-1.3.19-6 and later.