Bug 986857 - Disk Monitoring not checking filesystem with logs
Summary: Disk Monitoring not checking filesystem with logs
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: 389-ds-base
Version: 6.4
Hardware: Unspecified
OS: Unspecified
urgent
high
Target Milestone: rc
: ---
Assignee: Rich Megginson
QA Contact: Sankar Ramalingam
URL:
Whiteboard:
Depends On:
Blocks: 987850
TreeView+ depends on / blocked
 
Reported: 2013-07-22 09:15 UTC by Ján Rusnačko
Modified: 2013-11-21 21:11 UTC (History)
5 users (show)

Fixed In Version: 389-ds-base-1.2.11.15-22.el6
Doc Type: Bug Fix
Doc Text:
Cause: Improper processing of the feature's configuration settings (logging critical is off) Consequence: If logging is not set to critical, then logs directory is incorrectly not checked for disk space. Fix: Fix the logic when processing the configuration settings Result: The log directories are checked regardless if logging is to set to critical or not.
Clone Of:
Environment:
Last Closed: 2013-11-21 21:11:01 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2013:1653 normal SHIPPED_LIVE 389-ds-base bug fix update 2013-11-20 21:53:19 UTC

Description Ján Rusnačko 2013-07-22 09:15:44 UTC
Description of problem:
When nsslapd-disk-monitoring-logging-critical is set to "on", disk monitoring plugin monitors filesystem containing DS logs. If filesystem is running out of space, it triggers disk monitoring plugin.

However, when nsslapd-disk-monitoring-logging-critical is set to "off", disk monitoring plugin is not checking mentioned filesystem. So, in case when there is not enough space to store logs, disk monitoring is not triggered, even though deleting rotated logfiles would help.

Version-Release number of selected component (if applicable):
389-ds-base-1.2.11.15-17.el6_4.x86_64

How reproducible:
always

Steps to Reproduce:
1. Create a separate filesystem and mount at /var/log/dirsrv/slapd-dstet, configure disk monitoring plugin:

[jrusnack@dstet dstet]$ ldapsearch -D "cn=directory manager" -w Secret123 -b "cn=config" -s base | grep "nsslapd-disk-monitoring"
nsslapd-disk-monitoring: on
nsslapd-disk-monitoring-threshold: 30000000
nsslapd-disk-monitoring-grace-period: 1
nsslapd-disk-monitoring-logging-critical: off
[jrusnack@dstet dstet]$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg_dstet-lv_root
                      9.4G  7.3G  1.8G  81% /
tmpfs                 499M   76K  499M   1% /dev/shm
/dev/vda1             485M   33M  427M   8% /boot
/home/jrusnack/testfs
                       39M  4.5M   33M  13% /var/log/dirsrv/slapd-dstet
[jrusnack@dstet dstet]$ dd if=/dev/zero of=/var/log/dirsrv/slapd-dstet
slapd-dstet/     slapd-dstet.bak/ 
[jrusnack@dstet dstet]$ dd if=/dev/zero of=/var/log/dirsrv/slapd-dstet/foo bs=1M count=10
10+0 records in
10+0 records out
10485760 bytes (10 MB) copied, 0.0435152 s, 241 MB/s
[jrusnack@dstet dstet]$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg_dstet-lv_root
                      9.4G  7.3G  1.8G  81% /
tmpfs                 499M   76K  499M   1% /dev/shm
/dev/vda1             485M   33M  427M   8% /boot
/home/jrusnack/testfs
                       39M   15M   23M  40% /var/log/dirsrv/slapd-dstet

# filesystem containing logs is now filled below the threshold, but disk monitoring plugin is not triggered and no error messages present in the error log

2. Try filling / below the threshold:

[jrusnack@dstet dstet]$ dd if=/dev/zero of=/home/jrusnack/foo bs=1K count=1768160
1768160+0 records in
1768160+0 records out
1810595840 bytes (1.8 GB) copied, 46.3845 s, 39.0 MB/s
[jrusnack@dstet dstet]$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg_dstet-lv_root
                      9.4G  8.9G   23M 100% /
tmpfs                 499M   76K  499M   1% /dev/shm
/dev/vda1             485M   33M  427M   8% /boot
/home/jrusnack/testfs
                       39M   15M   23M  40% /var/log/dirsrv/slapd-dstet
[jrusnack@dstet dstet]$ sudo cat /var/log/dirsrv/slapd-dstet/errors | grep "is too low"
[22/Jul/2013:11:09:28 +0200] - Disk space is too low on disk (/), remaining space: 22996 Kb, disabling access and audit logging.
[22/Jul/2013:11:09:38 +0200] - Disk space is too low on disk (/), remaining space: 22996 Kb, deleting rotated logs.
[22/Jul/2013:11:09:58 +0200] - Disk space is too low on disk (/), remaining space: 22992 Kb

Actual results:
Disk monitoring plugin is not monitoring available space of /var/log/dirsrv/<instance name> mountpoint when nsslapd-disk-monitoring-logging-critical is set to "off".

Additional info:
See related bugs
Bug 982621 - Disk monitoring not shutting down with nsslapd-disk-monitoring-logging-critical set to off 
Bug 972930 - RHDS not shutting down when disk monitoring threshold is reached to half.

Comment 2 mreynolds 2013-07-22 15:49:30 UTC
This behaviour was by design, but it should be changed.

Comment 3 Nathan Kinder 2013-07-22 19:12:33 UTC
Upstream ticket:
https://fedorahosted.org/389/ticket/47441

Comment 7 Ján Rusnačko 2013-08-23 11:38:48 UTC
Tests are passing, verified on 389-ds-base-1.2.11.15-22.el6.x86_64

Comment 8 errata-xmlrpc 2013-11-21 21:11:01 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-1653.html


Note You need to log in before you can comment on or make changes to this bug.