Bug 765257 (GLUSTER-3525)

Summary:	self-heal of special files (fifo, character, block, and symlinks) does not correctly reset trusted.afr xattrs
Product:	[Community] GlusterFS	Reporter:	Joe Julian <joe>
Component:	replicate	Assignee:	Kaushal <kaushal>
Status:	CLOSED CURRENTRELEASE	QA Contact:
Severity:	medium	Docs Contact:
Priority:	medium
Version:	3.2.3	CC:	gluster-bugs, rodrigo, vijay
Target Milestone:	---
Target Release:	---
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:		Type:	---
Regression:	---	Mount Type:	fuse
Documentation:	---	CRM:
Verified Versions:	3.1.6, 3.2.3	Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:

Description Joe Julian 2011-09-07 18:53:13 UTC

I had a fifo that had been through previous versions of glusterfs. Apparently those earlier verisons had added the trusted.afr tags and left them in a dirty state.

Running a stat on that fifo started a background self-heal on it and completed that self-heal successfully. Since fifo's are no longer assigned trusted.afr xattrs, the old dirty ones remained. stat'ing the fifo again started a self-heal again.

The expected behavior would be to remove the no-longer-needed trusted.afr attributes.

Comment 1 Joe Julian 2011-09-08 19:52:25 UTC

[2011-09-08 15:45:26.472481] I [afr-common.c:790:afr_lookup_done] 0-home-replicate-2: background  data self-heal triggered. path: /jjulian/.icedteaplugin/icedtea-appletviewer-to-plugin
[2011-09-08 15:45:26.472594] I [afr-self-heal-common.c:1536:afr_self_heal_completion_cbk] 0-home-replicate-2: background  data self-heal completed on /jjulian/.icedteaplugin/icedtea-appletviewer-to-plugin
[2011-09-08 15:45:37.253118] I [afr-common.c:790:afr_lookup_done] 0-home-replicate-2: background  data self-heal triggered. path: /jjulian/.icedteaplugin/icedtea-appletviewer-to-plugin
[2011-09-08 15:45:37.253188] I [afr-self-heal-common.c:1536:afr_self_heal_completion_cbk] 0-home-replicate-2: background  data self-heal completed on /jjulian/.icedteaplugin/icedtea-appletviewer-to-plugin
[2011-09-08 15:46:46.888644] I [afr-common.c:790:afr_lookup_done] 0-home-replicate-2: background  data self-heal triggered. path: /jjulian/.icedteaplugin/icedtea-appletviewer-to-plugin
[2011-09-08 15:46:46.888752] I [afr-self-heal-common.c:1536:afr_self_heal_completion_cbk] 0-home-replicate-2: background  data self-heal completed on /jjulian/.icedteaplugin/icedtea-appletviewer-to-plugin
[2011-09-08 15:46:48.131477] I [afr-common.c:790:afr_lookup_done] 0-home-replicate-2: background  data self-heal triggered. path: /jjulian/.icedteaplugin/icedtea-appletviewer-to-plugin
[2011-09-08 15:46:48.131557] I [afr-self-heal-common.c:1536:afr_self_heal_completion_cbk] 0-home-replicate-2: background  data self-heal completed on /jjulian/.icedteaplugin/icedtea-appletviewer-to-plugin
[2011-09-08 15:46:49.304426] I [afr-common.c:790:afr_lookup_done] 0-home-replicate-2: background  data self-heal triggered. path: /jjulian/.icedteaplugin/icedtea-appletviewer-to-plugin
[2011-09-08 15:46:49.304508] I [afr-self-heal-common.c:1536:afr_self_heal_completion_cbk] 0-home-replicate-2: background  data self-heal completed on /jjulian/.icedteaplugin/icedtea-appletviewer-to-plugin
[2011-09-08 15:46:50.443775] I [afr-common.c:790:afr_lookup_done] 0-home-replicate-2: background  data self-heal triggered. path: /jjulian/.icedteaplugin/icedtea-appletviewer-to-plugin
[2011-09-08 15:46:50.443879] I [afr-self-heal-common.c:1536:afr_self_heal_completion_cbk] 0-home-replicate-2: background  data self-heal completed on /jjulian/.icedteaplugin/icedtea-appletviewer-to-plugin


This was done by just repeatedly stat'ing this fifo.

The dirty bits (clean bits are omitted in my script) both before and after "stat"ing the fifo are:
./c_home/jjulian/.icedteaplugin/icedtea-appletviewer-to-plugin 
	trusted.afr.home-client-6	0x000100000000
	trusted.afr.home-client-8	0x000100000000

# gluster volume info home

Volume Name: home
Type: Distributed-Replicate
Status: Started
Number of Bricks: 4 x 3 = 12
Transport-type: tcp
Bricks:
Brick1: ewcs2:/var/spool/glusterfs/a_home
Brick2: ewcs4:/var/spool/glusterfs/a_home
Brick3: ewcs7:/var/spool/glusterfs/a_home
Brick4: ewcs2:/var/spool/glusterfs/b_home
Brick5: ewcs4:/var/spool/glusterfs/b_home
Brick6: ewcs7:/var/spool/glusterfs/b_home
Brick7: ewcs2:/var/spool/glusterfs/c_home
Brick8: ewcs4:/var/spool/glusterfs/c_home
Brick9: ewcs7:/var/spool/glusterfs/c_home
Brick10: ewcs2:/var/spool/glusterfs/d_home
Brick11: ewcs4:/var/spool/glusterfs/d_home
Brick12: ewcs7:/var/spool/glusterfs/d_home
Options Reconfigured:
performance.quick-read: on
performance.io-cache: off

Comment 2 Joe Julian 2011-09-08 19:53:25 UTC

The same thing happens for character and block devices as well.

Comment 3 Joe Julian 2011-09-22 11:40:41 UTC

1. Create, start, and mount a replica 2 volume (/mnt/afrtest)
2. kill one of the brick processes
3. mkfifo /mnt/afrtest/fubar

At this point, {brick directory}/fubar has no trusted.afr xattr

4. bring back up the downed brick
5. stat /mnt/afrtest/fubar
6. getfattr {brick directory}/fubar

# file: fubar
trusted.afr.afrtest-client-1=0x000000010000000100000000
trusted.gfid=...

Comment 4 Anand Avati 2011-10-31 09:52:54 UTC

CHANGE: http://review.gluster.com/647 (Fixes self-heal of special files like device files, fifo files, socket files) merged in release-3.2 by Vijay Bellur (vijay)

Comment 5 Anand Avati 2011-10-31 09:53:27 UTC

CHANGE: http://review.gluster.com/618 (Fixes self-heal of special files like device files, fifo files, socket files) merged in master by Vijay Bellur (vijay)