Bug 1005663 - DHT : directory selfheal is not creating extended attribute trusted.glusterfs.dht after creating directory [NEEDINFO]
DHT : directory selfheal is not creating extended attribute trusted.glusterfs...
Status: CLOSED ERRATA
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: glusterfs (Show other bugs)
2.1
x86_64 Linux
medium Severity medium
: ---
: RHGS 2.1.2
Assigned To: Vijaikumar Mallikarjuna
amainkar
: Regression, ZStream
Depends On:
Blocks: 1047331
  Show dependency treegraph
 
Reported: 2013-09-09 01:59 EDT by Rachana Patel
Modified: 2016-05-11 18:47 EDT (History)
8 users (show)

See Also:
Fixed In Version: glusterfs-3.4.0.53rhs
Doc Type: Bug Fix
Doc Text:
Previously, upon a directory lookup, error logs were generated if trusted.glusterfs.dht extended attribute was missing. With this fix, error logs are not generated for directories with no layouts.
Story Points: ---
Clone Of:
: 1047331 (view as bug list)
Environment:
Last Closed: 2014-02-25 02:38:03 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
racpatel: needinfo? (vmallika)


Attachments (Terms of Use)

  None (edit)
Description Rachana Patel 2013-09-09 01:59:06 EDT
Description of problem:
DHT : directory selfheal is not creatting extended attribute trusted.glusterfs.dht after creating directory ,because of that on lookup 'disk layout missing',' mismatching layouts' messages are coming in log

Version-Release number of selected component (if applicable):
3.4.0.32rhs-1.el6_4.x86_64

How reproducible:
always

Steps to Reproduce:
1.creatd DHT volume and FUSE mount it
2.one brick is down ; create dir and files

[root@DHT1 1]# gluster v status
Status of volume: testdht
Gluster process						Port	Online	Pid
------------------------------------------------------------------------------
Brick 10.70.37.195:/rhs/brick1/1			N/A	N	435
Brick 10.70.37.195:/rhs/brick1/2			49167	Y	2048
Brick 10.70.37.98:/rhs/brick1/1				49154	Y	29098
NFS Server on localhost					2049	Y	2327
NFS Server on 10.70.37.98				2049	Y	29111
 
There are no active volume tasks

mount:-
[root@rhs-client22 dhttest]# cd up1
[root@rhs-client22 up1]# mkdir down{1..10}
mkdir: cannot create directory `down1': Transport endpoint is not connected
mkdir: cannot create directory `down2': Transport endpoint is not connected
mkdir: cannot create directory `down4': Transport endpoint is not connected
mkdir: cannot create directory `down5': Transport endpoint is not connected
mkdir: cannot create directory `down10': Transport endpoint is not connected
[root@rhs-client22 up1]# cd down1
-bash: cd: down1: No such file or directory
[root@rhs-client22 up1]# cd down3
[root@rhs-client22 down3]# touch down{1..10}
[root@rhs-client22 down3]# cd ..
[root@rhs-client22 up1]# touch fdown{1..10}
touch: cannot touch `fdown6': Transport endpoint is not connected
touch: cannot touch `fdown9': Transport endpoint is not connected

3. make that brick up and from mount point issue lookup on that dir. 
self heal should create directory on sub-vol which was down previously and also set xattr.

[root@DHT1 1]# gluster v status
Status of volume: testdht
Gluster process						Port	Online	Pid
------------------------------------------------------------------------------
Brick 10.70.37.195:/rhs/brick1/1			49166	Y	435
Brick 10.70.37.195:/rhs/brick1/2			49167	Y	2048
Brick 10.70.37.98:/rhs/brick1/1				49154	Y	29098
NFS Server on localhost					2049	Y	2327
NFS Server on 10.70.37.98				2049	Y	29111
 
There are no active volume tasks

mount:-
[root@rhs-client22 up1]# ls -l down3

log:-
[2013-09-09 03:54:19.773681] I [dht-layout.c:720:dht_layout_dir_mismatch] 0-testdht-dht: /up1/down3 - disk layout missing
[2013-09-09 03:54:19.773766] I [dht-common.c:638:dht_revalidate_cbk] 0-testdht-dht: mismatching layouts for /up1/down3


backend:-

previously down   <-----------------------------------------
[root@DHT1 1]# getfattr -d -m . -e hex /rhs/brick1/1/up1/down3
getfattr: Removing leading '/' from absolute path names
# file: rhs/brick1/1/up1/down3
trusted.gfid=0xc052b59c4d644d0f9002136f83975e10

[root@DHT1 1]# getfattr -d -m . -e hex /rhs/brick1/2/up1/down3
# file: rhs/brick1/2/up1/down3
trusted.gfid=0xc052b59c4d644d0f9002136f83975e10
trusted.glusterfs.dht=0x00000001000000007fffffffffffffff
[root@DHT3 1]# getfattr -d -m . -e hex /rhs/brick1/*/up1/down3
getfattr: Removing leading '/' from absolute path names
# file: rhs/brick1/1/up1/down3
trusted.gfid=0xc052b59c4d644d0f9002136f83975e10
trusted.glusterfs.dht=0x0000000100000000000000007ffffffe


Actual results:
self heal is creating dir on previously down brick but it is not setting xattr for the same

Expected results:
self heal should create dir and set xattr for the same

Additional info:
In Anshi release it was creating dir on down sub-vol and was setting all xattr for the same
Comment 5 Vivek Agarwal 2013-12-26 08:44:45 EST
setting corbett and other flags as this is being tracked for corbett.
Comment 6 Vijaikumar Mallikarjuna 2013-12-26 09:56:23 EST
xattr are not set because there is no layout created for a directory.
Current workaround is to run fix-layout and rebalance from the CLI.
Comment 7 Vijaikumar Mallikarjuna 2013-12-30 07:50:59 EST
In Anshi release, xattrs was written with 0-0% hash range when there is no layout for the directory.

Current behavior is, we ignore the directories when the layout is absent and do not write xattrs. 
This change is introduced by the patch: http://review.gluster.org/4668/

Logging for missed layout need to be fixed when err == 0, and start == stop.
Comment 8 Vijaikumar Mallikarjuna 2013-12-30 08:22:37 EST
Patch: https://code.engineering.redhat.com/gerrit/#/c/17922/
Comment 9 Pavithra 2014-01-03 02:11:19 EST
Please verify the edited doc text for technical accuracy.
Comment 10 Vijaikumar Mallikarjuna 2014-01-03 02:24:33 EST
Doc-text looks good to me.
Comment 11 Rachana Patel 2014-01-07 04:29:40 EST
verified with 3.4.0.53rhs-1.el6rhs.x86_64

self heal is creating dir on previously down brick and also set xattr for the same.

Hence moving it to verified
Comment 14 errata-xmlrpc 2014-02-25 02:38:03 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHEA-2014-0208.html

Note You need to log in before you can comment on or make changes to this bug.