Description of problem: The log rotation script installed by 3.3.1-ubuntu1~precise7 moves logfiles but leaves gluster writing to the old file. This results in .1 (or whatever) files being written to and 0-length log files which should contain the latest log info. For example, on my system gluster has installed a file called 'glusterfs-common' that looks like this: /var/log/glusterfs/*.log { daily rotate 7 delaycompress compress notifempty missingok } That's broken. A similar script that works properly looks something like this: /var/log/glusterfs/*.log { missingok rotate 3 weekly compress postrotate /usr/bin/killall -HUP glusterfs > /dev/null 2>&1 || true /usr/bin/killall -HUP glusterd > /dev/null 2>&1 || true endscript } Version-Release number of selected component (if applicable): 3.3.1-ubuntu1~precise7 How reproducible: Always broken! Steps to Reproduce: 1. Wait a week 2. Look in /var/log/glusterfs 3. Additional info: /var/log/glusterfs/*.log { daily rotate 7 delaycompress compress notifempty missingok } That's broken. A similar script that works properly (and which I'm using) would look something like this: /var/log/glusterfs/*.log { daily rotate 7 delaycompress compress notifempty missingok postrotate /usr/bin/killall -HUP glusterfs > /dev/null 2>&1 || true /usr/bin/killall -HUP glusterd > /dev/null 2>&1 || true endscript }
Hm, sorry for the double-paste there.
You should probably HUP the 'glusterfsd' processes (brick export daemons) as well since they also generate log files.
Just installed glusterfs-client (+common) 3.4.5 into Debian Wheezy and the same problem still exists, the logrotate only HUPs glusterd while on the client-side it should kick glusterfs process. Also the server version only HUPs the glusterd, thus we never rotate glusterfs nor glusterfsd logs properly but filehandle follows the rotated file and eventually (after a week, "daily, rotate 7" by default) we won't be logging anymore. Grepping around the source tree i found a somewhat (better) working log rotation file from extras directory: debian/glusterfs-server.logrotate: [ ! -f /var/run/glusterd.pid ] || kill -HUP `cat /var/run/glusterd.pid` debian/glusterfs-common.logrotate: [ ! -f /var/run/glusterd.pid ] || kill -HUP `cat /var/run/glusterd.pid` extras/glusterfs-logrotate: /usr/bin/killall -HUP glusterfs > /dev/null 2>&1 || true extras/glusterfs-logrotate: /usr/bin/killall -HUP glusterd > /dev/null 2>&1 || true extras/glusterfs-logrotate: /usr/bin/killall -HUP glusterfsd > /dev/null 2>&1 || true https://github.com/gluster/glusterfs/blob/master/extras/glusterfs-logrotate
I think it is fixed in master now. I can see we are HUPing for glusterfs, glusterd and glusterfsd # Rotate client logs /var/log/glusterfs/*.log { sharedscripts weekly rotate 52 missingok compress delaycompress notifempty postrotate /usr/bin/killall -HUP glusterfs > /dev/null 2>&1 || true /usr/bin/killall -HUP glusterd > /dev/null 2>&1 || true endscript } # Rotate server logs /var/log/glusterfs/bricks/*.log { sharedscripts weekly rotate 52 missingok compress delaycompress notifempty postrotate /usr/bin/killall -HUP glusterfsd > /dev/null 2>&1 || true endscript } https://github.com/gluster/glusterfs/blob/master/extras/glusterfs-logrotate
*** This bug has been marked as a duplicate of bug 1126788 ***
*** This bug has been marked as a duplicate of bug 763785 ***