Bug 1126801

Summary: glusterfs logrotate config file pollutes global config
Product: [Community] GlusterFS Reporter: Niels de Vos <ndevos>
Component: loggingAssignee: Lalatendu Mohanty <lmohanty>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: high Docs Contact:
Priority: medium    
Version: 3.5.3CC: bugs, gluster-bugs, joey, lmohanty, martin.x.andersen, olim, plgs
Target Milestone: ---Keywords: EasyFix, Patch, Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: glusterfs-3.5.3 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-11-21 16:01:44 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1126802, 1158923    
Bug Blocks: 1126788, 1157160    

Description Niels de Vos 2014-08-05 10:28:23 UTC
+++ This bug was initially created as a clone of Bug #1126788 +++
+++                                                           +++
+++ This bug is for the Community Gluster version 3.5.2       +++

Description of problem:

Glusterfs's logrotate config file in the /etc/logrotate.d directory pollutes logrotate's global config file at /etc/logrotate.conf.  Namely, the file /etc/logrotate.d/glusterfs-georep has global directives in the beginning of the file that reads

"""

rotate 52
missingok

compress
delaycompress
notifempty

"""

When this file is included into logrotate's config file (line 18 of /etc/logrotate.conf, which reads "include /etc/logrotate.d"), it is splashed into the state of logrotate's config reader, "as if it was included inline" (man logrotate).  Therefore, those lines will override whatever corresponding configuration terms already present in /etc/logrotate.conf.

Especially, the "rotate 52" directive will delay the removal of old logs up to a year globally (!!), resulting in a large number of old logs that should have been deleted.

Please move the above directives inside the curly braces so they don't pollute the global config state.


Version-Release number of selected component (if applicable):

glusterfs-3.5.2-1.fc20


How reproducible:

Very.


Steps to Reproduce:

1.  Install glusterfs.
2.  Watch old logs grow;  output of logrotate -d /etc/logrotate.conf showing "52 rotations" not just for glusterfs georeplication logs but almost all other logs.


Actual results:

Many old logs are not removed.  logrotate -d /etc/logrotate.conf output shows "52 rotations" for all logs processed after glusterfs-georep that don't override the "rotate" directive individually.


Expected results:

Should not have interfered with other logs.


Additional info:

The spurious old logs caused by this still needs manual removal after the fix, because logrotate, given "rotate 4" directive, won't even touch beyond the 5th log file.

Comment 1 Lalatendu Mohanty 2014-08-06 11:47:45 UTC
Working in it.

Comment 2 Lalatendu Mohanty 2014-10-29 12:49:50 UTC
Patch for master is at http://review.gluster.org/8994

Comment 3 Anand Avati 2014-10-29 19:43:25 UTC
REVIEW: http://review.gluster.org/9001 (logrotate: gluster logrotate config should not be global) posted (#1) for review on release-3.5 by Lalatendu Mohanty (lmohanty)

Comment 4 Anand Avati 2014-10-30 09:19:18 UTC
COMMIT: http://review.gluster.org/9001 committed in release-3.5 by Niels de Vos (ndevos) 
------
commit a970fc0cd7ec4ddb806273b0d7279c52587452a4
Author: Lalatendu Mohanty <lmohanty>
Date:   Thu Oct 30 01:11:02 2014 +0530

    logrotate: gluster logrotate config should not be global
    
    Issue : Previously glusterfs logrotate config file pollutes
    global config. So moved the directives inside the curly braces,
     so they don't pollute the global config state.
    
    Change-Id: I8836893dfcdf457d9c5d766612d687bfce64e2ae
    BUG: 1126801
    "Signed-off-by: Lalatendu Mohanty <lmohanty>"
    "Reviewed-on: http://review.gluster.org/8994"
    "Reviewed-by: Niels de Vos <ndevos>"
    "Tested-by: Gluster Build System <jenkins.com>"
    (cherry picked from commit a5d73daabf6df95bc73b186d92f3e2d1239a6f8a)
    Signed-off-by: Lalatendu Mohanty <lmohanty>
    Reviewed-on: http://review.gluster.org/9001
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Niels de Vos <ndevos>

Comment 5 Niels de Vos 2014-11-05 09:24:14 UTC
The second Beta for GlusterFS 3.5.3 has been released [1]. Please verify if the release solves this bug report for you. In case the glusterfs-3.5.3beta2 release does not have a resolution for this issue, leave a comment in this bug and move the status to ASSIGNED. If this release fixes the problem for you, leave a note and change the status to VERIFIED.

Packages for several distributions have been made available on [2] to make testing easier.

[1] http://supercolony.gluster.org/pipermail/gluster-users/2014-November/019359.html
[2] http://download.gluster.org/pub/gluster/glusterfs/qa-releases/3.5.3beta2/

Comment 6 Niels de Vos 2014-11-21 16:01:44 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.5.3, please reopen this bug report.

glusterfs-3.5.3 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://supercolony.gluster.org/pipermail/announce/2014-November/000042.html
[2] http://supercolony.gluster.org/pipermail/gluster-users/

Comment 7 Martin Andersen 2015-08-12 12:42:01 UTC
This issue is not resolved. In the latest version for RHEL 7.1 (3.6.0.29-2.el7), the glusterfs-specific directives are still present outside their main block, thus affecting *all* logs system-wide that do not have an explicit rotation frequency set (why is it that the promise of a new release is regularly assumed to resolve existing bugs without actual verification of such?) ;)

$ cat /etc/logrotate.d/glusterfs

# perform the log rotate every week
weekly
# keep the backup of 52 weeks
rotate 52
missingok

# compress the logs, but from the .2 onwards
compress
delaycompress
notifempty

# Rotate client logs
/var/log/glusterfs/*.log {
  sharedscripts
  postrotate
  /usr/bin/killall -HUP glusterfs > /dev/null 2>&1 || true
  /usr/bin/killall -HUP glusterd > /dev/null 2>&1 || true
  endscript
}


$ rpm -qa|grep gluster
glusterfs-3.6.0.29-2.el7.x86_64
glusterfs-fuse-3.6.0.29-2.el7.x86_64
glusterfs-libs-3.6.0.29-2.el7.x86_64
glusterfs-api-3.6.0.29-2.el7.x86_64

Comment 8 Joe Pruett 2015-12-17 17:58:13 UTC
and more annoying is that qemu has dependencies on gluster, so this problam shows up in more than just gluster installations.

Comment 9 Niels de Vos 2015-12-17 20:24:47 UTC
(In reply to Joe Pruett from comment #8)
> and more annoying is that qemu has dependencies on gluster, so this problam
> shows up in more than just gluster installations.

This is the upstream Gluster Community bug, and it definitely has been fixed there. You need to open a support case with on the Red Hat Customer Portal https://access.redhat.com/ so that the RHEL package can get fixed with a backport of the change.

Comment 10 Joe Pruett 2015-12-17 20:29:46 UTC
sorry, just venting. there is a bug against rhel-6 about this issue.