Bug 1294059

Summary: Lot of Inode not found messages in glfsheal log file
Product: Red Hat Gluster Storage Reporter: Bhaskarakiran <byarlaga>
Component: disperseAssignee: Pranith Kumar K <pkarampu>
Status: CLOSED CURRENTRELEASE QA Contact: Matt Zywusko <mzywusko>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: rhgs-3.1CC: aspandey, mzywusko, pkarampu, rhs-bugs
Target Milestone: ---Keywords: ZStream
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: glusterfs-3.8rc2 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1294743 (view as bug list) Environment:
Last Closed: 2017-08-17 08:19:48 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On:    
Bug Blocks: 1294743, 1302943    

Description Bhaskarakiran 2015-12-24 10:47:04 UTC
Description of problem:
=======================

Seeing lot of inode not found messages in glfsheal log file.

This is on a 4+2 ec volume and 2x2 tier attached to it. Was running brick down and heal info commands

[2015-12-24 10:23:47.397946] W [inode.c:646:inode_new] (-->/usr/sbin/glfsheal(+0x2ab1) [0x7f36f0c27ab1] -->/lib64/libglusterfs.so.0(syncop_gfid_to_path+0x5c) [0x7f36f07bd6fc] -->/lib64/libglusterfs.so.0(inode_new+0x8c) [0x7f36f077bf4c] ) 0-dpvol-disperse-0: inode not found
[2015-12-24 10:23:47.398344] W [inode.c:646:inode_new] (-->/usr/sbin/glfsheal(+0x2ab1) [0x7f36f0c27ab1] -->/lib64/libglusterfs.so.0(syncop_gfid_to_path+0x5c) [0x7f36f07bd6fc] -->/lib64/libglusterfs.so.0(inode_new+0x8c) [0x7f36f077bf4c] ) 0-dpvol-disperse-0: inode not found
[2015-12-24 10:23:47.398544] W [inode.c:646:inode_new] (-->/usr/sbin/glfsheal(+0x2ab1) [0x7f36f0c27ab1] -->/lib64/libglusterfs.so.0(syncop_gfid_to_path+0x5c) [0x7f36f07bd6fc] -->/lib64/libglusterfs.so.0(inode_new+0x8c) [0x7f36f077bf4c] ) 0-dpvol-disperse-0: inode not found
[2015-12-24 10:23:47.398748] W [inode.c:646:inode_new] (-->/usr/sbin/glfsheal(+0x2ab1) [0x7f36f0c27ab1] -->/lib64/libglusterfs.so.0(syncop_gfid_to_path+0x5c) [0x7f36f07bd6fc] -->/lib64/libglusterfs.so.0(inode_new+0x8c) [0x7f36f077bf4c] ) 0-dpvol-disperse-0: inode not found
[2015-12-24 10:23:47.399003] W [inode.c:646:inode_new] (-->/usr/sbin/glfsheal(+0x2ab1) [0x7f36f0c27ab1] -->/lib64/libglusterfs.so.0(syncop_gfid_to_path+0x5c) [0x7f36f07bd6fc] -->/lib64/libglusterfs.so.0(inode_new+0x8c) [0x7f36f077bf4c] ) 0-dpvol-disperse-0: inode not found
[2015-12-24 10:23:47.399204] W [inode.c:646:inode_new] (-->/usr/sbin/glfsheal(+0x2ab1) [0x7f36f0c27ab1] -->/lib64/libglusterfs.so.0(syncop_gfid_to_path+0x5c) [0x7f36f07bd6fc] -->/lib64/libglusterfs.so.0(inode_new+0x8c) [0x7f36f077bf4c] ) 0-dpvol-disperse-0: inode not found
[2015-12-24 10:23:47.399392] W [inode.c:646:inode_new] (-->/usr/sbin/glfsheal(+0x2ab1) [0x7f36f0c27ab1] -->/lib64/libglusterfs.so.0(syncop_gfid_to_path+0x5c) [0x7f36f07bd6fc] -->/lib64/libglusterfs.so.0(inode_new+0x8c) [0x7f36f077bf4c] ) 0-dpvol-disperse-0: inode not found
[2015-12-24 10:23:47.399579] W [inode.c:646:inode_new] (-->/usr/sbin/glfsheal(+0x2ab1) [0x7f36f0c27ab1] -->/lib64/libglusterfs.so.0(syncop_gfid_to_path+0x5c) [0x7f36f07bd6fc] -->/lib64/libglusterfs.so.0(inode_new+0x8c) [0x7f36f077bf4c] ) 0-dpvol-disperse-0: inode not found

[root@transformers ~]# gluster v info dpvol
 
Volume Name: dpvol
Type: Tier
Volume ID: 0bf13965-09df-4f06-a15d-fb16e69c9cf5
Status: Started
Number of Bricks: 10
Transport-type: tcp
Hot Tier :
Hot Tier Type : Distributed-Replicate
Number of Bricks: 2 x 2 = 4
Brick1: tettnang:/rhs/brick2/dpvol-ht4
Brick2: tettnang:/rhs/brick2/dpvol-ht3
Brick3: tettnang:/rhs/brick2/dpvol-ht2
Brick4: tettnang:/rhs/brick2/dpvol-ht1
Cold Tier:
Cold Tier Type : Disperse
Number of Bricks: 1 x (4 + 2) = 6
Brick5: vertigo:/rhs/brick1/dpvol1
Brick6: ninja:/rhs/brick1/dpvol2
Brick7: rhs-client18:/rhs/brick1/dpvol3
Brick8: rhs-client19:/rhs/brick1/dpvol4
Brick9: transformers:/rhs/brick1/dpvol5
Brick10: interstellar:/rhs/brick1/dpvol6
Options Reconfigured:
cluster.tier-mode: cache
features.ctr-enabled: on
performance.readdir-ahead: on
features.uss: on
features.quota: on
features.inode-quota: on
features.quota-deem-statfs: on
client.event-threads: 4
server.event-threads: 4
[root@transformers ~]# 



Version-Release number of selected component (if applicable):
=============================================================
3.7.5-13

[root@transformers ~]# gluster --version
glusterfs 3.7.5 built on Dec 22 2015 19:44:25
Repository revision: git://git.gluster.com/glusterfs.git
Copyright (c) 2006-2011 Gluster Inc. <http://www.gluster.com>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GlusterFS under the terms of the GNU General Public License.
[root@transformers ~]# 

How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:
================
sosreports will be attached.

Comment 4 Mike McCune 2016-03-28 23:18:47 UTC
This bug was accidentally moved from POST to MODIFIED via an error in automation, please see mmccune with any questions