Description of problem: DHT : directory selfheal is not creatting extended attribute trusted.glusterfs.dht after creating directory ,because of that on lookup 'disk layout missing',' mismatching layouts' messages are coming in log Version-Release number of selected component (if applicable): 3.4.0.32rhs-1.el6_4.x86_64 How reproducible: always Steps to Reproduce: 1.creatd DHT volume and FUSE mount it 2.one brick is down ; create dir and files [root@DHT1 1]# gluster v status Status of volume: testdht Gluster process Port Online Pid ------------------------------------------------------------------------------ Brick 10.70.37.195:/rhs/brick1/1 N/A N 435 Brick 10.70.37.195:/rhs/brick1/2 49167 Y 2048 Brick 10.70.37.98:/rhs/brick1/1 49154 Y 29098 NFS Server on localhost 2049 Y 2327 NFS Server on 10.70.37.98 2049 Y 29111 There are no active volume tasks mount:- [root@rhs-client22 dhttest]# cd up1 [root@rhs-client22 up1]# mkdir down{1..10} mkdir: cannot create directory `down1': Transport endpoint is not connected mkdir: cannot create directory `down2': Transport endpoint is not connected mkdir: cannot create directory `down4': Transport endpoint is not connected mkdir: cannot create directory `down5': Transport endpoint is not connected mkdir: cannot create directory `down10': Transport endpoint is not connected [root@rhs-client22 up1]# cd down1 -bash: cd: down1: No such file or directory [root@rhs-client22 up1]# cd down3 [root@rhs-client22 down3]# touch down{1..10} [root@rhs-client22 down3]# cd .. [root@rhs-client22 up1]# touch fdown{1..10} touch: cannot touch `fdown6': Transport endpoint is not connected touch: cannot touch `fdown9': Transport endpoint is not connected 3. make that brick up and from mount point issue lookup on that dir. self heal should create directory on sub-vol which was down previously and also set xattr. [root@DHT1 1]# gluster v status Status of volume: testdht Gluster process Port Online Pid ------------------------------------------------------------------------------ Brick 10.70.37.195:/rhs/brick1/1 49166 Y 435 Brick 10.70.37.195:/rhs/brick1/2 49167 Y 2048 Brick 10.70.37.98:/rhs/brick1/1 49154 Y 29098 NFS Server on localhost 2049 Y 2327 NFS Server on 10.70.37.98 2049 Y 29111 There are no active volume tasks mount:- [root@rhs-client22 up1]# ls -l down3 log:- [2013-09-09 03:54:19.773681] I [dht-layout.c:720:dht_layout_dir_mismatch] 0-testdht-dht: /up1/down3 - disk layout missing [2013-09-09 03:54:19.773766] I [dht-common.c:638:dht_revalidate_cbk] 0-testdht-dht: mismatching layouts for /up1/down3 backend:- previously down <----------------------------------------- [root@DHT1 1]# getfattr -d -m . -e hex /rhs/brick1/1/up1/down3 getfattr: Removing leading '/' from absolute path names # file: rhs/brick1/1/up1/down3 trusted.gfid=0xc052b59c4d644d0f9002136f83975e10 [root@DHT1 1]# getfattr -d -m . -e hex /rhs/brick1/2/up1/down3 # file: rhs/brick1/2/up1/down3 trusted.gfid=0xc052b59c4d644d0f9002136f83975e10 trusted.glusterfs.dht=0x00000001000000007fffffffffffffff [root@DHT3 1]# getfattr -d -m . -e hex /rhs/brick1/*/up1/down3 getfattr: Removing leading '/' from absolute path names # file: rhs/brick1/1/up1/down3 trusted.gfid=0xc052b59c4d644d0f9002136f83975e10 trusted.glusterfs.dht=0x0000000100000000000000007ffffffe Actual results: self heal is creating dir on previously down brick but it is not setting xattr for the same Expected results: self heal should create dir and set xattr for the same Additional info: In Anshi release it was creating dir on down sub-vol and was setting all xattr for the same
setting corbett and other flags as this is being tracked for corbett.
xattr are not set because there is no layout created for a directory. Current workaround is to run fix-layout and rebalance from the CLI.
In Anshi release, xattrs was written with 0-0% hash range when there is no layout for the directory. Current behavior is, we ignore the directories when the layout is absent and do not write xattrs. This change is introduced by the patch: http://review.gluster.org/4668/ Logging for missed layout need to be fixed when err == 0, and start == stop.
Patch: https://code.engineering.redhat.com/gerrit/#/c/17922/
Please verify the edited doc text for technical accuracy.
Doc-text looks good to me.
verified with 3.4.0.53rhs-1.el6rhs.x86_64 self heal is creating dir on previously down brick and also set xattr for the same. Hence moving it to verified
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHEA-2014-0208.html
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days