Bug 1078061

Summary:	Need ability to heal mismatching user extended attributes without any changelogs
Product:	[Community] GlusterFS	Reporter:	Pranith Kumar K <pkarampu>
Component:	replicate	Assignee:	Pranith Kumar K <pkarampu>
Status:	CLOSED CURRENTRELEASE	QA Contact:
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	mainline	CC:	gluster-bugs
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	glusterfs-3.5.1beta	Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:
Clones:	1134691 (view as bug list)		Environment:
Last Closed:	2014-06-24 11:03:45 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1134691, 1144274

Description Pranith Kumar K 2014-03-19 06:27:43 UTC

Description of problem:

For snapshot design it was decided that frequent operations like 'cp -a <large-dir>' should not pause while taking snapshots. This can lead to metadata/xattr mismatches without any afr changelogs because those fops are not going to be blocked while taking snapshots on different bricks. Afr at the time of lookup detects metadata mismatches in stat structures and initiates a self-heal. But if there are any xattr mismatches between two replicas then there is no capability at the moment to detect and heal them. This enhancement tracks the changes required to achieve this change.

For afr to detect there are mismatches in xattrs, it should be able to query i.e. listxattr in lookup itself, to avoid extra over the network listxattr call latencies.
This requires changes in posix xlator.

Afr should use this feature in posix to 'listxattr' and see if there are any mismatches and trigger self-heals on that file probably at the time of fresh-lookup. This behaviour can be exposed in volumes that are exported as part of snapshot.

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 1 Anand Avati 2014-03-19 06:31:21 UTC

REVIEW: http://review.gluster.org/7293 (storage/posix: add list xattr capability to lookup) posted (#1) for review on master by Pranith Kumar Karampuri (pkarampu)

Comment 2 Anand Avati 2014-03-21 09:36:32 UTC

REVIEW: http://review.gluster.org/7293 (storage/posix: add list xattr capability to lookup) posted (#2) for review on master by Pranith Kumar Karampuri (pkarampu)

Comment 3 Anand Avati 2014-04-10 12:06:23 UTC

REVIEW: http://review.gluster.org/7434 (storage/posix: add list xattr capability to lookup) posted (#1) for review on release-3.5 by Pranith Kumar Karampuri (pkarampu)

Comment 4 Anand Avati 2014-04-10 20:35:08 UTC

REVIEW: http://review.gluster.org/7434 (storage/posix: add list xattr capability to lookup) posted (#2) for review on release-3.5 by Pranith Kumar Karampuri (pkarampu)

Comment 5 Anand Avati 2014-04-11 06:57:12 UTC

REVIEW: http://review.gluster.org/7444 (cluster/afr: perform list-xattr during lookup) posted (#1) for review on release-3.5 by Ravishankar N (ravishankar)

Comment 6 Anand Avati 2014-04-15 08:26:01 UTC

REVIEW: http://review.gluster.org/7434 (storage/posix: add list xattr capability to lookup) posted (#3) for review on release-3.5 by Pranith Kumar Karampuri (pkarampu)

Comment 7 Anand Avati 2014-04-15 12:51:48 UTC

REVIEW: http://review.gluster.org/7293 (storage/posix: add list xattr capability to lookup) posted (#3) for review on master by Pranith Kumar Karampuri (pkarampu)

Comment 8 Anand Avati 2014-04-21 09:23:44 UTC

REVIEW: http://review.gluster.org/7444 (cluster/afr: perform list-xattr during lookup) posted (#2) for review on release-3.5 by Ravishankar N (ravishankar)

Comment 9 Anand Avati 2014-04-28 16:53:50 UTC

COMMIT: http://review.gluster.org/7434 committed in release-3.5 by Vijay Bellur (vbellur) 
------
commit e49e56c5e81e7e6a4cc4d0ac2982446dd1887ef9
Author: Pranith Kumar K <pkarampu>
Date:   Wed Mar 19 07:52:48 2014 +0530

    storage/posix: add list xattr capability to lookup
    
    BUG: 1078061
    Change-Id: Ib84f40c84e2d66c86b1cec4a7cceb5aea189ba4a
    Signed-off-by: Pranith Kumar K <pkarampu>
    Reviewed-on: http://review.gluster.org/7434
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Xavier Hernandez <xhernandez>
    Reviewed-by: Ravishankar N <ravishankar>
    Reviewed-by: Vijay Bellur <vbellur>

Comment 10 Anand Avati 2014-04-28 16:56:16 UTC

COMMIT: http://review.gluster.org/7444 committed in release-3.5 by Vijay Bellur (vbellur) 
------
commit 0d2dbd569f5ee187fc7b864c30cc668fee465bb1
Author: Ravishankar N <ravishankar>
Date:   Fri Apr 11 05:45:56 2014 +0000

    cluster/afr: perform list-xattr during lookup
    
    Detect and heal mismatching user extended attributes during lookup.
    
    Depends on: http://review.gluster.org/#/c/7434/
    
    Change-Id: I49410aafd319ac159fdf9e6f9201871bbf2f67bd
    BUG: 1078061
    Signed-off-by: Ravishankar N <ravishankar>
    Reviewed-on: http://review.gluster.org/7444
    Reviewed-by: Pranith Kumar Karampuri <pkarampu>
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Vijay Bellur <vbellur>

Comment 11 Anand Avati 2014-04-29 09:19:10 UTC

REVIEW: http://review.gluster.org/7293 (storage/posix: add list xattr capability to lookup) posted (#4) for review on master by Pranith Kumar Karampuri (pkarampu)

Comment 12 Anand Avati 2014-04-29 21:16:58 UTC

COMMIT: http://review.gluster.org/7293 committed in master by Anand Avati (avati) 
------
commit 91ab65f812ec3b674f53230eacbd0d71964d3d01
Author: Pranith Kumar K <pkarampu>
Date:   Wed Mar 19 07:52:48 2014 +0530

    storage/posix: add list xattr capability to lookup
    
    BUG: 1078061
    Change-Id: Ie26d28b8a74aa0d1eceff14a84c3cd3e302dcdb5
    Signed-off-by: Pranith Kumar K <pkarampu>
    Reviewed-on: http://review.gluster.org/7293
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Anand Avati <avati>

Comment 13 Niels de Vos 2014-05-25 09:07:21 UTC

The first (and last?) Beta for GlusterFS 3.5.1 has been released [1]. Please verify if the release solves this bug report for you. In case the glusterfs-3.5.1beta release does not have a resolution for this issue, leave a comment in this bug and move the status to ASSIGNED. If this release fixes the problem for you, leave a note and change the status to VERIFIED.

Packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update (possibly an "updates-testing" repository) infrastructure for your distribution.

[1] http://supercolony.gluster.org/pipermail/gluster-users/2014-May/040377.html
[2] http://supercolony.gluster.org/pipermail/gluster-users/

Comment 14 Niels de Vos 2014-06-24 11:03:45 UTC

This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.5.1, please reopen this bug report.

glusterfs-3.5.1 has been announced on the Gluster Users mailinglist [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://supercolony.gluster.org/pipermail/gluster-users/2014-June/040723.html
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user