Bug 852147

Summary:	glusterd operations hang if the other peers are down
Product:	[Red Hat Storage] Red Hat Gluster Storage	Reporter:	Vidya Sakar <vinaraya>
Component:	glusterd	Assignee:	krishnan parthasarathi <kparthas>
Status:	CLOSED ERRATA	QA Contact:	Sachidananda Urs <surs>
Severity:	unspecified	Docs Contact:
Priority:	high
Version:	2.0	CC:	amarts, gluster-bugs, nsathyan, pkarampu, racpatel, rfortier, rhs-bugs, spandura, vbellur
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	glusterfs-3.4.0.2rhs-1.el6rhs	Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:	847214	Environment:
Last Closed:	2013-09-23 22:38:58 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	847214
Bug Blocks:	858476

Description Vidya Sakar 2012-08-27 17:34:55 UTC

+++ This bug was initially created as a clone of Bug #847214 +++

Description of problem:
Did a volume set operation while the other peers in the cluster were down. Op-sm hung.
Op-sm is stuck in an infinite state-transition:
Old State: [Ack drain]
New State: [Ack drain]
Event    : [GD_OP_EVENT_START_UNLOCK]
timestamp: [2012-08-10 06:10:25]

Old State: [Ack drain]
New State: [Ack drain]
Event    : [GD_OP_EVENT_START_UNLOCK]
timestamp: [2012-08-10 06:10:28]

Old State: [Ack drain]
New State: [Ack drain]
Event    : [GD_OP_EVENT_START_UNLOCK]
timestamp: [2012-08-10 06:10:28]

Old State: [Ack drain]
New State: [Ack drain]
Event    : [GD_OP_EVENT_START_UNLOCK]
timestamp: [2012-08-10 06:10:31]

Old State: [Ack drain]
New State: [Ack drain]
Event    : [GD_OP_EVENT_START_UNLOCK]
timestamp: [2012-08-10 06:10:31]


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
This the setup in which I got the problem, but I think it can be triggered even with 2 machines
1.Have a cluster with 3 machines.
2.Bring two of the glusterds down.
3.Execute any glusterd operation command which uses op-sm, I used volume set. 
  
Actual results:
The operation will hang after commit-op.
Expected results:
volume set operation should have been successful.

Additional info:

Comment 2 Amar Tumballi 2012-11-29 08:46:46 UTC

*** Bug 861919 has been marked as a duplicate of this bug. ***

Comment 3 Amar Tumballi 2012-11-29 08:46:54 UTC

*** Bug 860568 has been marked as a duplicate of this bug. ***

Comment 4 Amar Tumballi 2012-12-28 08:57:44 UTC

http://review.gluster.org/4297 is a base need for this to get fixed. Will fix 'all' the commands once this gets fixed for volume status.

Comment 5 krishnan parthasarathi 2013-01-09 07:55:48 UTC

*** Bug 852295 has been marked as a duplicate of this bug. ***

Comment 6 Vijay Bellur 2013-02-03 19:53:12 UTC

CHANGE: http://review.gluster.org/4295 (glusterd: Moved node rsp functions to glusterd-utils.c) merged in master by Anand Avati (avati)

Comment 7 Vijay Bellur 2013-02-03 19:54:30 UTC

CHANGE: http://review.gluster.org/4296 (glusterd: Added syncop version of BRICK_OP) merged in master by Anand Avati (avati)

Comment 8 Vijay Bellur 2013-02-03 19:56:31 UTC

CHANGE: http://review.gluster.org/4297 (glusterd: Made volume-status use synctask framework) merged in master by Anand Avati (avati)

Comment 9 Vijay Bellur 2013-02-08 20:03:56 UTC

CHANGE: http://review.gluster.org/4494 (glusterd: Made volume-statedump use synctask framework.) merged in master by Anand Avati (avati)

Comment 10 Vijay Bellur 2013-02-08 21:29:12 UTC

CHANGE: http://review.gluster.org/4492 (glusterd: Made volume-delete use synctask framework.) merged in master by Anand Avati (avati)

Comment 11 Vijay Bellur 2013-02-08 21:29:30 UTC

CHANGE: http://review.gluster.org/4491 (glusterd: Made volume-stop use synctask framework.) merged in master by Anand Avati (avati)

Comment 12 Vijay Bellur 2013-02-08 22:08:36 UTC

CHANGE: http://review.gluster.org/4490 (glusterd : Made volume clear-locks use synctask framework.) merged in master by Anand Avati (avati)

Comment 13 Vijay Bellur 2013-02-08 22:21:28 UTC

CHANGE: http://review.gluster.org/4489 (glusterd: Made volume-sync use synctask framework.) merged in master by Anand Avati (avati)

Comment 14 Vijay Bellur 2013-02-09 03:04:47 UTC

CHANGE: http://review.gluster.org/4488 (glusterd : Made volume-set use synctask framework.) merged in master by Anand Avati (avati)

Comment 15 Vijay Bellur 2013-02-09 03:06:04 UTC

CHANGE: http://review.gluster.org/4474 (glusterd: Making volume-reset use synctask framework) merged in master by Anand Avati (avati)

Comment 16 Vijay Bellur 2013-02-09 03:09:43 UTC

CHANGE: http://review.gluster.org/4473 (glusterd: Made gsync set use synctask framework) merged in master by Anand Avati (avati)

Comment 17 Vijay Bellur 2013-02-14 01:47:21 UTC

CHANGE: http://review.gluster.org/4478 (glusterd: Made log-rotate use synctask framework.) merged in master by Anand Avati (avati)

Comment 18 Vijay Bellur 2013-02-17 04:41:25 UTC

CHANGE: http://review.gluster.org/4495 (glusterd: Made volume-quota use synctask framework.) merged in master by Anand Avati (avati)

Comment 19 Vijay Bellur 2013-02-17 06:32:36 UTC

CHANGE: http://review.gluster.org/4493 (glusterd: Made volume-heal use synctask framework.) merged in master by Anand Avati (avati)

Comment 20 Vijay Bellur 2013-02-20 02:58:01 UTC

CHANGE: http://review.gluster.org/4507 (glusterd: Made gd_synctask_begin less 'monolithic' in terms of LOC.) merged in master by Anand Avati (avati)

Comment 24 Sachidananda Urs 2013-08-02 05:24:24 UTC

Verified on: glusterfs 3.4.0.14rhs built on Jul 30 2013 09:09:36

Comment 25 Scott Haines 2013-09-23 22:38:58 UTC

Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. 

For information on the advisory, and where to find the updated files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-1262.html

Comment 26 Scott Haines 2013-09-23 22:41:32 UTC

Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. 

For information on the advisory, and where to find the updated files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-1262.html