Bug 1144274

Summary:	Need ability to heal mismatching user extended attributes without any changelogs
Product:	[Community] GlusterFS	Reporter:	Ravishankar N <ravishankar>
Component:	replicate	Assignee:	Ravishankar N <ravishankar>
Status:	CLOSED CURRENTRELEASE	QA Contact:
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	3.6.0	CC:	bugs, gluster-bugs, pkarampu
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	glusterfs-3.6.0beta1	Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:	1134691	Environment:
Last Closed:	2014-11-11 08:39:19 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1078061, 1134691
Bug Blocks:

Description Ravishankar N 2014-09-19 06:42:49 UTC

+++ This bug was initially created as a clone of Bug #1134691 +++

+++ This bug was initially created as a clone of Bug #1078061 +++

Description of problem:

For snapshot design it was decided that frequent operations like 'cp -a <large-dir>' should not pause while taking snapshots. This can lead to metadata/xattr mismatches without any afr changelogs because those fops are not going to be blocked while taking snapshots on different bricks. Afr at the time of lookup detects metadata mismatches in stat structures and initiates a self-heal. But if there are any xattr mismatches between two replicas then there is no capability at the moment to detect and heal them. This enhancement tracks the changes required to achieve this change.

For afr to detect there are mismatches in xattrs, it should be able to query i.e. listxattr in lookup itself, to avoid extra over the network listxattr call latencies.
This requires changes in posix xlator.

Afr should use this feature in posix to 'listxattr' and see if there are any mismatches and trigger self-heals on that file probably at the time of fresh-lookup. This behaviour can be exposed in volumes that are exported as part of snapshot.


--- Additional comment from Anand Avati on 2014-09-06 01:19:38 EDT ---

COMMIT: http://review.gluster.org/8558 committed in master by Pranith Kumar Karampuri (pkarampu) 
------
commit 15a2088da508539a292f2a1863377dc40d264c8c
Author: Ravishankar N <ravishankar>
Date:   Wed Sep 3 20:49:53 2014 +0000

    cluster/afr: perform list-xattr during lookup
    
    Detect and heal mismatching user extended attributes during lookup.
    'Forward' port of http://review.gluster.org/#/c/7444/
    
    Change-Id: Id03c9746f083ffd3014711d0b3a2e5a71a45eed4
    BUG: 1134691
    Signed-off-by: Ravishankar N <ravishankar>
    Reviewed-on: http://review.gluster.org/8558
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Pranith Kumar Karampuri <pkarampu>
    Tested-by: Pranith Kumar Karampuri <pkarampu>

Comment 1 Anand Avati 2014-09-19 06:44:08 UTC

REVIEW: http://review.gluster.org/8773 (cluster/afr: perform list-xattr during lookup) posted (#1) for review on release-3.6 by Ravishankar N (ravishankar)

Comment 2 Anand Avati 2014-09-19 18:02:28 UTC

COMMIT: http://review.gluster.org/8773 committed in release-3.6 by Vijay Bellur (vbellur) 
------
commit 61bf585398c514d437a022792f6726a4292509dd
Author: Ravishankar N <ravishankar>
Date:   Wed Sep 3 20:49:53 2014 +0000

    cluster/afr: perform list-xattr during lookup
    
    Detect and heal mismatching user extended attributes during lookup.
    Backport of: http://review.gluster.org/8558
    
    Change-Id: Id03c9746f083ffd3014711d0b3a2e5a71a45eed4
    BUG: 1144274
    Signed-off-by: Ravishankar N <ravishankar>
    Reviewed-on: http://review.gluster.org/8773
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Pranith Kumar Karampuri <pkarampu>
    Reviewed-by: Vijay Bellur <vbellur>

Comment 3 Niels de Vos 2014-09-22 12:46:34 UTC

A beta release for GlusterFS 3.6.0 has been released. Please verify if the release solves this bug report for you. In case the glusterfs-3.6.0beta1 release does not have a resolution for this issue, leave a comment in this bug and move the status to ASSIGNED. If this release fixes the problem for you, leave a note and change the status to VERIFIED.

Packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update (possibly an "updates-testing" repository) infrastructure for your distribution.

[1] http://supercolony.gluster.org/pipermail/gluster-users/2014-September/018836.html
[2] http://supercolony.gluster.org/pipermail/gluster-users/

Comment 4 Niels de Vos 2014-11-11 08:39:19 UTC

This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.6.1, please reopen this bug report.

glusterfs-3.6.1 has been announced [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://supercolony.gluster.org/pipermail/gluster-users/2014-November/019410.html
[2] http://supercolony.gluster.org/mailman/listinfo/gluster-users