Bug 1227028

Summary:	nfs-ganesha: Discrepancies with lock states recovery during migration
Product:	[Community] GlusterFS	Reporter:	Kaleb KEITHLEY <kkeithle>
Component:	common-ha	Assignee:	Kaleb KEITHLEY <kkeithle>
Status:	CLOSED CURRENTRELEASE	QA Contact:
Severity:	high	Docs Contact:
Priority:	high
Version:	3.7.0	CC:	jthottan, kkeithle, mmadhusu, ndevos, skoduri
Target Milestone:	---	Keywords:	Triaged
Target Release:	---
Hardware:	All
OS:	All
Whiteboard:
Fixed In Version:	glusterfs-3.7.2	Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:	1219485	Environment:
Last Closed:	2015-06-20 09:49:21 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1216039, 1219485
Bug Blocks:	1227206

Description Kaleb KEITHLEY 2015-06-01 18:18:59 UTC

+++ This bug was initially created as a clone of Bug #1219485 +++

+++ This bug was initially created as a clone of Bug #1216039 +++

Description of problem:

We have seen below issue while testing open/lock states recovery after IP failover in nfs-ganesha HA setup - 

The cluster has not been put into grace before IP fails over. There seem to be a timing issue there. "pcs constraint colocation order" seem to have not completely solved it.

--- Additional comment from Kaleb KEITHLEY on 2015-05-07 09:08:27 EDT ---

yes, there's a race between setting grace and failing over the virt IP.

See http://review.gluster.org/10490

--- Additional comment from Anand Avati on 2015-05-07 09:26:01 EDT ---

REVIEW: http://review.gluster.org/10646 (common-ha: fix race between setting grace and virt IP fail-over) posted (#1) for review on master by Kaleb KEITHLEY (kkeithle)

--- Additional comment from Anand Avati on 2015-05-26 14:50:40 EDT ---

REVIEW: http://review.gluster.org/10646 (common-ha: fix race between setting grace and virt IP fail-over) posted (#2) for review on master by Kaleb KEITHLEY (kkeithle)

--- Additional comment from Anand Avati on 2015-06-01 14:13:51 EDT ---

COMMIT: http://review.gluster.org/10646 committed in master by Kaleb KEITHLEY (kkeithle) 
------
commit 751c4583bbaa59ebfe492ab6ecfab3108711f4c5
Author: Kaleb S. KEITHLEY <kkeithle>
Date:   Thu May 7 09:22:04 2015 -0400

    common-ha: fix race between setting grace and virt IP fail-over
    
    Also send stderr output of `pcs resource {create,delete} $node-dead_ip-1`
    to /dev/null to avoid flooding the logs
    
    Change-Id: I29d526429cc4d7521971cd5e2e69bfb64bfc5ca9
    BUG: 1219485
    Signed-off-by: Kaleb S. KEITHLEY <kkeithle>
    Reviewed-on: http://review.gluster.org/10646
    Tested-by: NetBSD Build System <jenkins.org>
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: soumya k <skoduri>
    Reviewed-by: Meghana M <mmadhusu>

Comment 1 Anand Avati 2015-06-01 18:28:42 UTC

REVIEW: http://review.gluster.org/11036 (common-ha: fix race between setting grace and virt IP fail-over) posted (#1) for review on release-3.7 by Kaleb KEITHLEY (kkeithle)

Comment 2 Anand Avati 2015-06-04 14:36:52 UTC

COMMIT: http://review.gluster.org/11036 committed in release-3.7 by Kaleb KEITHLEY (kkeithle) 
------
commit 4c262454d538e57f8c86d0e4defad759605b870a
Author: Kaleb S. KEITHLEY <kkeithle>
Date:   Thu May 7 09:22:04 2015 -0400

    common-ha: fix race between setting grace and virt IP fail-over
    
    Also send stderr output of `pcs resource {create,delete} $node-dead_ip-1`
    to /dev/null to avoid flooding the logs
    
    > backport of http://review.gluster.org/#/c/10646/, BZ 1219485
    
    Change-Id: I29d526429cc4d7521971cd5e2e69bfb64bfc5ca9
    BUG: 1227028
    Signed-off-by: Kaleb S. KEITHLEY <kkeithle>
    Reviewed-on: http://review.gluster.org/10646
    Tested-by: NetBSD Build System <jenkins.org>
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: soumya k <skoduri>
    Reviewed-by: Meghana M <mmadhusu>
    Reviewed-on: http://review.gluster.org/11036

Comment 3 Kaleb KEITHLEY 2015-06-05 14:36:09 UTC

*** Bug 1216039 has been marked as a duplicate of this bug. ***

Comment 4 Niels de Vos 2015-06-20 09:49:21 UTC

This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.7.2, please reopen this bug report.

glusterfs-3.7.2 has been announced on the Gluster Packaging mailinglist [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://www.gluster.org/pipermail/packaging/2015-June/000006.html
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user