Bug 1100563

Summary:	No log messages available for the split brain case in glusterfs
Product:	[Red Hat Storage] Red Hat Gluster Storage	Reporter:	Shubhendu Tripathi <shtripat>
Component:	replicate	Assignee:	Pranith Kumar K <pkarampu>
Status:	CLOSED CURRENTRELEASE	QA Contact:	storage-qa-internal <storage-qa-internal>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	rhgs-3.0	CC:	kaushal, nsathyan, ravishankar, rhs-bugs, sabose, storage-qa-internal
Target Milestone:	---	Keywords:	FutureFeature
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Enhancement
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2016-01-14 09:31:47 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1109690
Bug Blocks:	1081900

Description Shubhendu Tripathi 2014-05-23 05:53:33 UTC

Description of problem:
There is no log messages available for the split brain case in glusterfs

Version-Release number of selected component (if applicable):


How reproducible:
Always

Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 2 Ravishankar N 2014-09-05 06:53:43 UTC

All important log messages in AFR have been converted to use the new logging framework. Split-brain related messages now appear in the gluster logs (either client mount log or gluster selfheal daemon log file). The message ID for split-brain related messages is 108008

Comment 3 Kaushal 2016-01-14 07:19:27 UTC

Please close this if the fix has already made it into a RHS/RHGS release. It should be in looking at the date of comment#2.

Comment 4 Ravishankar N 2016-01-14 09:31:47 UTC

Fix is there in RHGS. Here's a sample output from the mount logs on RHGS-3.1.2:
--------------------------
[2016-01-14 09:30:36.003926] I [MSGID: 108031] [afr-common.c:1881:afr_local_discovery_cbk] 0-testvol-replicate-0: selecting local read_child testvol-client-0
[2016-01-14 09:30:36.004154] I [MSGID: 108031] [afr-common.c:1881:afr_local_discovery_cbk] 0-testvol-replicate-0: selecting local read_child testvol-client-1
[2016-01-14 09:30:36.013367] E [MSGID: 108008] [afr-read-txn.c:89:afr_read_txn_refresh_done] 0-testvol-replicate-0: Failing READ on gfid 2da7f1c5-6676-4f8b-8d91-f0baa6280aeb: split-brain observed. [Input/output error]
[2016-01-14 09:30:36.013601] W [fuse-bridge.c:2218:fuse_readv_cbk] 0-glusterfs-fuse: 48: READ => -1 (Input/output error)
[2016-01-14 09:30:36.015218] W [MSGID: 108008] [afr-read-txn.c:250:afr_read_txn] 0-testvol-replicate-0: Unreadable subvolume -1 found with event generation 6 for gfid 2da7f1c5-6676-4f8b-8d91-f0baa6280aeb. (Possible split-brain)
[2016-01-14 09:30:36.019438] E [MSGID: 108008] [afr-read-txn.c:89:afr_read_txn_refresh_done] 0-testvol-replicate-0: Failing READ on gfid 2da7f1c5-6676-4f8b-8d91-f0baa6280aeb: split-brain observed. [Input/output error]
[2016-01-14 09:30:36.019596] W [fuse-bridge.c:2218:fuse_readv_cbk] 0-glusterfs-fuse: 49: READ => -1 (Input/output error)

---------------------------------------
Closing the bug.