Bug 1294053 - Excessive logging in mount when bricks of the replica are down
Excessive logging in mount when bricks of the replica are down
Status: CLOSED CURRENTRELEASE
Product: GlusterFS
Classification: Community
Component: replicate (Show other bugs)
mainline
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Ravishankar N
: Triaged
Depends On:
Blocks: 1299724
  Show dependency treegraph
 
Reported: 2015-12-24 05:11 EST by Ravishankar N
Modified: 2016-06-16 09:52 EDT (History)
1 user (show)

See Also:
Fixed In Version: glusterfs-3.8rc2
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1299724 (view as bug list)
Environment:
Last Closed: 2016-06-16 09:52:16 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Ravishankar N 2015-12-24 05:11:18 EST
Description of problem:

Commit 2b7226f9d3470d8fe4c98c1fddb06e7f641e364d did not check for the
validity of a dict before doing a dict_get. This floods the mount log with lot of spurios messages when one of the bricks is down in a replica subovolume:

[2015-12-24 09:31:48.793530] I [dict.c:462:dict_get] (-->/usr/local/lib/glusterfs/3.8dev/xlator/cluster/replicate.so(+0x4de19) [0x7fabffdd6e19] -->/usr/local/lib/glusterfs/3.8dev/xlator/cluster/replicate.so(+0x4d946) [0x7fabffdd6946] -->/usr/local/lib/libglusterfs.so.0(dict_get+0x72) [0x7fac11800302] ) 0-dict: !this || key=glusterfs.bad-inode [Invalid argument]
[2015-12-24 09:31:48.794164] I [dict.c:462:dict_get] (-->/usr/local/lib/glusterfs/3.8dev/xlator/cluster/replicate.so(+0x4de19) [0x7fabffdd6e19] -->/usr/local/lib/glusterfs/3.8dev/xlator/cluster/replicate.so(+0x4d946) [0x7fabffdd6946] -->/usr/local/lib/libglusterfs.so.0(dict_get+0x72) [0x7fac11800302] ) 0-dict: !this || key=glusterfs.bad-inode [Invalid argument]
[2015-12-24 09:31:48.794692] I [dict.c:462:dict_get] (-->/usr/local/lib/glusterfs/3.8dev/xlator/cluster/replicate.so(+0x4de19) [0x7fabffdd6e19]
Comment 1 Vijay Bellur 2015-12-24 05:12:23 EST
REVIEW: http://review.gluster.org/13077 (afr: Fix excessive logging in afr_accuse_smallfiles()) posted (#1) for review on master by Ravishankar N (ravishankar@redhat.com)
Comment 2 Vijay Bellur 2015-12-28 06:55:47 EST
REVIEW: http://review.gluster.org/13077 (afr: Fix excessive logging in afr_accuse_smallfiles()) posted (#2) for review on master by Ravishankar N (ravishankar@redhat.com)
Comment 3 Vijay Bellur 2015-12-29 02:26:33 EST
COMMIT: http://review.gluster.org/13077 committed in master by Pranith Kumar Karampuri (pkarampu@redhat.com) 
------
commit 4c1b66720121ca97ccfac3e09f93519f2f07ddfd
Author: Ravishankar N <ravishankar@redhat.com>
Date:   Thu Dec 24 15:22:05 2015 +0530

    afr: Fix excessive logging in afr_accuse_smallfiles()
    
    Commit 2b7226f9d3470d8fe4c98c1fddb06e7f641e364d did not check for the
    validity of a dict before doing a dict_get. Fix that.
    
    Change-Id: Ie21f4da19256b17196f242cd8fd5bb76b0a69c1e
    BUG: 1294053
    Signed-off-by: Ravishankar N <ravishankar@redhat.com>
    Reviewed-on: http://review.gluster.org/13077
    Tested-by: NetBSD Build System <jenkins@build.gluster.org>
    Tested-by: Gluster Build System <jenkins@build.gluster.com>
    Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Comment 4 Ravishankar N 2016-01-06 08:32:31 EST
Found one more code path where the message gets logged, hence moving the bug back to post and using the same bug ID for the next patch.
Comment 5 Vijay Bellur 2016-01-06 08:37:05 EST
REVIEW: http://review.gluster.org/13185 (afr : Check if dict is valid in afr_replies_interpret()) posted (#1) for review on master by Ravishankar N (ravishankar@redhat.com)
Comment 6 Vijay Bellur 2016-01-10 23:32:27 EST
REVIEW: http://review.gluster.org/13185 (afr : Check if dict is valid in afr_replies_interpret()) posted (#2) for review on master by Ravishankar N (ravishankar@redhat.com)
Comment 7 Vijay Bellur 2016-01-12 05:42:54 EST
REVIEW: http://review.gluster.org/13185 (afr : Check if dict is valid in afr_replies_interpret()) posted (#3) for review on master by Ravishankar N (ravishankar@redhat.com)
Comment 8 Vijay Bellur 2016-01-18 02:12:01 EST
REVIEW: http://review.gluster.org/13185 (afr : Check if dict is valid in afr_replies_interpret()) posted (#4) for review on master by Ravishankar N (ravishankar@redhat.com)
Comment 9 Vijay Bellur 2016-01-21 04:48:29 EST
COMMIT: http://review.gluster.org/13185 committed in master by Pranith Kumar Karampuri (pkarampu@redhat.com) 
------
commit 30ef018c00b0d1be5d590c9518c159a7946d040e
Author: Ravishankar N <ravishankar@redhat.com>
Date:   Wed Jan 6 13:34:01 2016 +0000

    afr : Check if dict is valid in afr_replies_interpret()
    
    posix_mkdir does not send response xdata. So even though replies are
    valid, the response xdata dict is NULL. Check if dict is non-null in
    afr_replies_interpret before doing dict_get
    
    Signed-off-by: Ravishankar N <ravishankar@redhat.com>
    
    Change-Id: If543d68d8bfd2433519105839d5be106076cc276
    BUG: 1294053
    Reviewed-on: http://review.gluster.org/13185
    Tested-by: Ravishankar N <ravishankar@redhat.com>
    Smoke: Gluster Build System <jenkins@build.gluster.com>
    NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
    CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
    Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Comment 10 Niels de Vos 2016-06-16 09:52:16 EDT
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.8.0, please open a new bug report.

glusterfs-3.8.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://blog.gluster.org/2016/06/glusterfs-3-8-released/
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user

Note You need to log in before you can comment on or make changes to this bug.