Bug 1282378

Summary:	nfs+afr:Failing SETATTR on gfid split-brain observed. [Input/output error] with fileop test
Product:	[Red Hat Storage] Red Hat Gluster Storage	Reporter:	Saurabh <saujain>
Component:	replicate	Assignee:	Ravishankar N <ravishankar>
Status:	CLOSED DUPLICATE	QA Contact:	storage-qa-internal <storage-qa-internal>
Severity:	medium	Docs Contact:
Priority:	unspecified
Version:	rhgs-3.1	CC:	mzywusko, pkarampu, ravishankar, rhs-bugs, saujain, skoduri, storage-qa-internal
Target Milestone:	---	Keywords:	ZStream
Target Release:	---
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2016-01-12 15:41:08 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Saurabh 2015-11-16 09:09:50 UTC

Description of problem:

Execute fileop test and in the nfs.log we get lot of input/output errors

Version-Release number of selected component (if applicable):
glusterfs-3.7.5-5.el7rhgs.x86_64

How reproducible:
always

Steps to Reproduce:
1. create a volume of 6x2 type
2. mount the volume over vers=3
3. execute fileop test

Actual results:
the fileop test finishes, but the nfs.log reports loads of input/output error,
[2015-11-16 16:57:09.974247] E [MSGID: 108008] [afr-transaction.c:1981:afr_transaction] 0-vol0-replicate-0: Failing SETATTR on gfid 93fce9eb-10f2-4d6e-b524-4a5d5ad4a005: split-brain observed. [Input/output error]
[2015-11-16 16:57:09.974315] E [MSGID: 109031] [dht-linkfile.c:306:dht_linkfile_setattr_cbk] 0-vol0-dht: Failed to set attr uid/gid on /run819/fileop_L1_29/fileop_L1_29_L2_29/fileop_file_29_29_29 :<gfid:00000000-0000-0000-0000-000000000000>  [Input/output error]


Expected results:
no EIO and messages related to it are expected in logs.

Additional info:

Comment 2 Niels de Vos 2015-11-16 12:00:24 UTC

Looks more like an AFR issue to me, changing components.

Comment 3 Ravishankar N 2015-11-16 12:25:17 UTC

Hi Saurabh, Are the files actually in split-brain? Could you check the afr xattrs (getfattr -d -m . -e hex /brick(s)/file) and the output of `gluster vol heal volname info` and 'gluster v heal volname info split-brain`?

Comment 4 Saurabh 2015-11-17 13:01:08 UTC

I tried to reproduce the problem and as you can see that there are no files listed for split-brain

[root@nfs12 ~]#  gluster volume heal vol0 info 
Brick 10.70.46.8:/rhs/brick1/d1r1
Number of entries: 0

Brick 10.70.46.25:/rhs/brick1/d1r2
Number of entries: 0

Brick 10.70.46.27:/rhs/brick1/d2r1
Number of entries: 0

Brick 10.70.46.39:/rhs/brick1/d2r2
Number of entries: 0

[root@nfs12 ~]# 
[root@nfs12 ~]# 
[root@nfs12 ~]#  gluster volume heal vol0 info split-brain
Brick 10.70.46.8:/rhs/brick1/d1r1
Number of entries in split-brain: 0

Brick 10.70.46.25:/rhs/brick1/d1r2
Number of entries in split-brain: 0

Brick 10.70.46.27:/rhs/brick1/d2r1
Number of entries in split-brain: 0

Brick 10.70.46.39:/rhs/brick1/d2r2
Number of entries in split-brain: 0

Comment 8 Pranith Kumar K 2016-01-12 15:41:08 UTC


*** This bug has been marked as a duplicate of bug 1294452 ***