Bug 1305742

Summary:	Lot of assertion failures are seen in nfs logs with disperse volume
Product:	[Community] GlusterFS	Reporter:	Pranith Kumar K <pkarampu>
Component:	disperse	Assignee:	Pranith Kumar K <pkarampu>
Status:	CLOSED CURRENTRELEASE	QA Contact:
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	3.7.7	CC:	bugs
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	glusterfs-3.7.9	Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:	1300929	Environment:
Last Closed:	2016-04-19 07:25:16 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1300929
Bug Blocks:

Description Pranith Kumar K 2016-02-09 05:42:11 UTC

+++ This bug was initially created as a clone of Bug #1300929 +++

Description of problem:
Hi Pranith,

I see a lot of these messages in nfs.log.

[2016-01-20 12:15:45.595207] E [ec-common.c:1945:__ec_manager] (-->/usr/lib64/glusterfs/3.7.5/xlator/cluster/distribute.so(dht_access+0x177) [0x7f6e09f39f67] -->/usr/lib64/glusterfs/3.7.5/xlator/cluster/disperse.so(ec_gf_access+0x2c) [0x7f6e0a168a1c] -->/usr/lib64/glusterfs/3.7.5/xlator/cluster/disperse.so(__ec_manager+0x166) [0x7f6e0a16e5d6] ) 0-: Assertion failed: fop->state >= 0
[2016-01-20 12:15:45.595411] E [ec-common.c:1945:__ec_manager] (-->/usr/lib64/glusterfs/3.7.5/xlator/cluster/distribute.so(dht_access+0x177) [0x7f6e09f39f67] -->/usr/lib64/glusterfs/3.7.5/xlator/cluster/disperse.so(ec_gf_access+0x2c) [0x7f6e0a168a1c] -->/usr/lib64/glusterfs/3.7.5/xlator/cluster/disperse.so(__ec_manager+0x166) [0x7f6e0a16e5d6] ) 0-: Assertion failed: fop->state >= 0

[root@transformers ~]# grep -i "fop->state >= 0" /var/log/glusterfs/nfs.log |wc -l
222918


-Bhaskar


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

--- Additional comment from Vijay Bellur on 2016-01-22 01:50:33 EST ---

REVIEW: http://review.gluster.org/13278 (cluster/ec: Never return -ve state) posted (#1) for review on master by Pranith Kumar Karampuri (pkarampu)

--- Additional comment from Vijay Bellur on 2016-01-27 05:46:22 EST ---

REVIEW: http://review.gluster.org/13278 (cluster/ec: Never return -ve state) posted (#2) for review on master by Pranith Kumar Karampuri (pkarampu)

--- Additional comment from Vijay Bellur on 2016-02-05 03:34:26 EST ---

REVIEW: http://review.gluster.org/13278 (cluster/ec: Never return -ve state) posted (#3) for review on master by Xavier Hernandez (xhernandez)

--- Additional comment from Vijay Bellur on 2016-02-08 10:33:15 EST ---

COMMIT: http://review.gluster.org/13278 committed in master by Pranith Kumar Karampuri (pkarampu) 
------
commit 95597b87e32b8615ca63f8b0b3bb8d488272d95d
Author: Pranith Kumar K <pkarampu>
Date:   Fri Jan 22 12:11:40 2016 +0530

    cluster/ec: Never return -ve state
    
    Ec manager shouldn't return -ve states, but it is, fixed that.
    
    Change-Id: I3f97c6ba2dbf9da724e8e1ee9b2c9da73f40013d
    BUG: 1300929
    Signed-off-by: Pranith Kumar K <pkarampu>
    Reviewed-on: http://review.gluster.org/13278
    Tested-by: Xavier Hernandez <xhernandez>
    Smoke: Gluster Build System <jenkins.com>
    CentOS-regression: Gluster Build System <jenkins.com>
    NetBSD-regression: NetBSD Build System <jenkins.org>

Comment 1 Vijay Bellur 2016-02-09 05:43:57 UTC

REVIEW: http://review.gluster.org/13400 (cluster/ec: Never return -ve state) posted (#1) for review on release-3.7 by Pranith Kumar Karampuri (pkarampu)

Comment 2 Vijay Bellur 2016-02-15 07:20:43 UTC

COMMIT: http://review.gluster.org/13400 committed in release-3.7 by Xavier Hernandez (xhernandez) 
------
commit 87faa0d763e1c416f12becc84b4c62cf6d545d1d
Author: Pranith Kumar K <pkarampu>
Date:   Fri Jan 22 12:11:40 2016 +0530

    cluster/ec: Never return -ve state
    
    Ec manager shouldn't return -ve states, but it is, fixed that.
    
     >Change-Id: I3f97c6ba2dbf9da724e8e1ee9b2c9da73f40013d
     >BUG: 1300929
     >Signed-off-by: Pranith Kumar K <pkarampu>
     >Reviewed-on: http://review.gluster.org/13278
     >Tested-by: Xavier Hernandez <xhernandez>
     >Smoke: Gluster Build System <jenkins.com>
     >CentOS-regression: Gluster Build System <jenkins.com>
     >NetBSD-regression: NetBSD Build System <jenkins.org>
    
    BUG: 1305742
    Change-Id: I3c0db88b286afca355a8a9ec73589eca57710da6
    Signed-off-by: Pranith Kumar K <pkarampu>
    Reviewed-on: http://review.gluster.org/13400
    Smoke: Gluster Build System <jenkins.com>
    NetBSD-regression: NetBSD Build System <jenkins.org>
    CentOS-regression: Gluster Build System <jenkins.com>
    Reviewed-by: Xavier Hernandez <xhernandez>

Comment 3 Mike McCune 2016-03-28 23:18:47 UTC

This bug was accidentally moved from POST to MODIFIED via an error in automation, please see mmccune with any questions

Comment 4 Kaushal 2016-04-19 07:25:16 UTC

This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.7.9, please open a new bug report.

glusterfs-3.7.9 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] https://www.gluster.org/pipermail/gluster-users/2016-March/025922.html
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user