Hide Forgot
+++ This bug was initially created as a clone of Bug #1286036 +++ +++ This bug was initially created as a clone of Bug #863100 +++ Description of problem: DHT :- inconsistent 'custom extended attributes',uid and gid, Access permission (for directories) if User set/modifies it after bringing one or more sub-volume down Version-Release number of selected component (if applicable): 3.3.0.3rhs-32.el6rhs.x86_64 How reproducible: always Steps to Reproduce: 1. Create a Distributed volume having 3 or more sub-volumes on multiple server and start that volume. 2. Fuse Mount the volume from the client-1 using “mount -t glusterfs server:/<volume> <client-1_mount_point>” 3. From mount point create some dirs and files inside it. 4. Bring on sub-volume down [root@Rhs3 t1]# gluster volume status test Status of volume: test Gluster process Port Online Pid ------------------------------------------------------------------------------ Brick 10.70.35.81:/home/t1 24009 Y 18564 Brick 10.70.35.85:/home/t1 24211 Y 16174 Brick 10.70.35.86:/home/t1 24212 Y 2360 NFS Server on localhost 38467 Y 2366 NFS Server on 10.70.35.81 38467 Y 12929 NFS Server on 10.70.35.85 38467 Y 10226 [root@Rhs3 t1]# kill -9 2360 5. Custom atribute:- from mount point set custom attribute for directory and verify it on all server client [root@client test]# setfattr -n user.foo -v bar2 d1 [root@client test]# getfattr -n user.foo d1 # file: d1 user.foo="bar2" server1:- [root@Rhs1 t1]# getfattr -n user.foo d1 # file: d1 user.foo="bar2" server2:- [root@Rhs2 t1]# getfattr -n user.foo d1 # file: d1 user.foo="bar2" server3:- [root@Rhs3 t1]# getfattr -n user.foo d1 d1: user.foo: No such attribute 6.from mount point verify owner and group of dir and then modify it [root@client test]# stat d1 File: `d1' Size: 12 Blocks: 2 IO Block: 131072 directory Device: 15h/21d Inode: 10442536925251715313 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2012-10-04 12:10:57.006871636 +0530 Modify: 2012-10-04 12:10:57.006871636 +0530 Change: 2012-10-04 12:10:57.007864913 +0530 [root@client test]# chown u1 d1 [root@client test]# chgrp t1 d1 [root@client test]# stat d1 File: `d1' Size: 12 Blocks: 2 IO Block: 131072 directory Device: 15h/21d Inode: 10442536925251715313 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 500/ u1) Gid: ( 500/ t1) Access: 2012-10-04 12:10:57.006871636 +0530 Modify: 2012-10-04 12:10:57.006871636 +0530 Change: 2012-10-04 12:13:05.168865621 +0530 7.verify that it is updated on all sub-volume except the down one server1:- [root@Rhs1 t1]# stat d1 File: `d1' Size: 6 Blocks: 8 IO Block: 4096 directory Device: fc05h/64517d Inode: 403116740 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 500/ UNKNOWN) Gid: ( 500/ UNKNOWN) Access: 2012-10-04 12:10:57.006871636 +0530 Modify: 2012-10-04 12:10:57.006871636 +0530 Change: 2012-10-04 12:13:05.168865621 +0530 server2:- [root@Rhs2 t1]# stat d1 File: `d1' Size: 6 Blocks: 8 IO Block: 4096 directory Device: fc05h/64517d Inode: 134423062 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 500/ UNKNOWN) Gid: ( 500/ UNKNOWN) Access: 2012-10-04 12:10:56.807630951 +0530 Modify: 2012-10-04 12:10:56.807630951 +0530 Change: 2012-10-04 12:13:04.970323409 +0530 server3:- [root@Rhs3 t1]# stat d1 File: `d1' Size: 6 Blocks: 8 IO Block: 4096 directory Device: fc05h/64517d Inode: 402655089 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2012-10-04 12:10:51.424531338 +0530 Modify: 2012-10-04 12:10:51.424531338 +0530 Change: 2012-10-04 12:10:51.610007436 +0530 8.from mount point verify Dir permission and modify it client [root@client test]# stat d2 File: `d2' Size: 12 Blocks: 2 IO Block: 131072 directory Device: 15h/21d Inode: 9860248238918728119 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2012-10-04 12:10:57.281276686 +0530 Modify: 2012-10-04 12:10:57.281276686 +0530 Change: 2012-10-04 12:10:57.282865329 +0530 [root@client test]# chmod 444 d2 [root@client test]# stat d2 File: `d2' Size: 12 Blocks: 2 IO Block: 131072 directory Device: 15h/21d Inode: 9860248238918728119 Links: 2 Access: (0444/dr--r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2012-10-04 12:10:57.281276686 +0530 Modify: 2012-10-04 12:10:57.281276686 +0530 Change: 2012-10-04 15:39:42.509942805 +0530 9. verify that it is updated on all sub-volume except the down one server1 [root@Rhs1 t1]# stat d2 File: `d2' Size: 6 Blocks: 8 IO Block: 4096 directory Device: fc05h/64517d Inode: 134694359 Links: 2 Access: (0444/dr--r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2012-10-04 12:10:57.281276686 +0530 Modify: 2012-10-04 12:10:57.281276686 +0530 Change: 2012-10-04 15:39:42.509942805 +0530 server2 [root@Rhs1 t1]# stat d2 File: `d2' Size: 6 Blocks: 8 IO Block: 4096 directory Device: fc05h/64517d Inode: 134694359 Links: 2 Access: (0444/dr--r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2012-10-04 12:10:57.281276686 +0530 Modify: 2012-10-04 12:10:57.281276686 +0530 Change: 2012-10-04 15:39:42.509942805 +0530 server3:- [root@Rhs3 t1]# stat d1 File: `d1' Size: 6 Blocks: 8 IO Block: 4096 directory Device: fc05h/64517d Inode: 402655089 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2012-10-04 12:10:51.424531338 +0530 Modify: 2012-10-04 12:10:51.424531338 +0530 Change: 2012-10-04 12:10:51.610007436 +0530 10. now bring all sub-volumes up and perform lookup from client. Verify updated 'custom extended attributes',uid and gid, Access permission from client 11.verify access permission from sub-volume which was down previously [root@Rhs3 t1]# stat d2 File: `d2' Size: 6 Blocks: 8 IO Block: 4096 directory Device: fc05h/64517d Inode: 134219638 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2012-10-04 12:10:51.700461206 +0530 Modify: 2012-10-04 12:10:51.700461206 +0530 Change: 2012-10-04 12:10:51.759989493 +0530 verify gid and uid on sub-volume which was down previously [root@Rhs3 t1]# stat d1 File: `d1' Size: 6 Blocks: 8 IO Block: 4096 directory Device: fc05h/64517d Inode: 402655089 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2012-10-04 12:10:51.424531338 +0530 Modify: 2012-10-04 12:10:51.424531338 +0530 Change: 2012-10-04 12:10:51.610007436 +0530 Verify custom extended attributes for that directory. server3:- [root@Rhs3 t1]# getfattr -n user.foo d1 d1: user.foo: No such attribute Actual results: mount point shows modified values but on subvolumes values are not consistent Expected results: Once sub-volume is up it should update values for 'custom extended attributes',uid and gid, Access permission (for directories). Additional info: --- Additional comment from RHEL Product and Program Management on 2012-10-04 17:55:42 MVT --- Since this issue was entered in bugzilla, the release flag has been set to ? to ensure that it is properly evaluated for this release. --- Additional comment from shishir gowda on 2012-10-19 09:21:57 MVT --- There are multiple issues in this bug. 1. For user related xattrs, we can not handle healing as in dht, we would not be able to identify the correct copy. The work around for this is a subsequent setxattr for the same key, which will fix the xattr mis-match 2. UID/GID: A fix is in progress (862967) 3. mis-matching permission: Will investigate it respond back --- Additional comment from shishir gowda on 2012-10-23 14:07:48 MVT --- Fix @ https://code.engineering.redhat.com/gerrit/#/c/150/ --- Additional comment from Amar Tumballi on 2013-02-15 17:11:59 MVT --- https://code.engineering.redhat.com/gerrit/#/c/1895/ --- Additional comment from Rachana Patel on 2013-03-19 18:09:46 MVT --- (In reply to comment #2) > There are multiple issues in this bug. > 1. For user related xattrs, we can not handle healing as in dht, we would > not be able to identify the correct copy. The work around for this is a > subsequent setxattr for the same key, which will fix the xattr mis-match > > 2. UID/GID: A fix is in progress (862967) > 3. mis-matching permission: Will investigate it respond back 1. If it is the case then, it should get documented. 2. It is dependent of bug 86296 and for that buf fixed in version is glusterfs-3.4.0qa5. So the same fix is available in latest build? 3. Could you please update on 3rd issue? what is the decision? --- Additional comment from Scott Haines on 2013-09-27 22:07:27 MVT --- Targeting for 3.0.0 (Denali) release. --- Additional comment from errata-xmlrpc on 2014-04-10 05:20:34 MVT --- This bug has been dropped from advisory RHEA-2014:17485 by Scott Haines (shaines@redhat.com) --- Additional comment from Susant Kumar Palai on 2014-05-22 11:04:45 MVT --- Here is a observation on the side effect of the patch in upstream : http://review.gluster.org/#/c/6983/. This patch works for all cases except one corner case. Currently we take the "permission info" for healing from a brick only if it has a layout xattr. Let say we add a new brick to a volume(newly added brick will not have a layout xattr) and all except the newly added brick goes down. Then we change the permission of the root directory. Now only the new brick has witnessed the new permission. If we bring all the bricks up, the older permission will be healed across all bricks as we dont take permission info if a brick does not have layout xattr. Here is the demo: brick1 brick permission(initial) 755 755 t0 UP ADDED BRICK t1 CHILD_DOWN t2 CHANGE PERMISSION LET SAY 777 on mount point t3 CHILD_UP t4 Heal 755 to all bricks as only this brick has layout xattr Final Permission after healing t5 755 755 -----> should have 777 $ Why not having a layout xattr for root of the brick for a newly added brick helps ? # If we assign a layout on root for the newly added brick, then as it will have the latest ctime it may corrupt the permission for the volume. example: brick1 brick2 brick3 t0 777(perm*) 777(perm*) Added as a brick will have perm* 755 by default t1 If it has layout & higher ctime, 755 will be healed for all Final Permission will be 755 instead of 777(bad) t2 755 755 755 Hence, creating a zeroed layout for root will create the above problem. $ Why healing on revalidate path is choosen ? # The reason we don't do metadata healing on fresh lookup path is, because directory selfheal is carried out on lookup path. Hence, once we do selfheal of directory on fresh lookup, we follow revalidate_cbk. And we will not be able to heal permission for that directory. So healing on the revalidate path is choosen. --- Additional comment from RHEL Product and Program Management on 2014-05-26 14:01:14 MVT --- This bug report previously had all acks and release flag approved. However since at least one of its acks has been changed, the release flag has been reset to ? by the bugbot (pm-rhel). The ack needs to become approved before the release flag can become approved again. --- Additional comment from Scott Haines on 2014-06-08 23:35:12 MVT --- Per engineering management on 06/06/2014, moving back to rhs-future backlog. --- Additional comment from errata-xmlrpc on 2014-06-20 14:20:27 MVT --- This bug has been dropped from advisory RHEA-2014:17485 by Vivek Agarwal (vagarwal@redhat.com) --- Additional comment from Nagaprasad Sathyanarayana on 2015-03-26 17:32:49 MVT --- After having triaged, it was agreed by all leads that this BZ can not be fixed for 3.1.0 release. --- Additional comment from John Skeoch on 2015-04-20 05:22:54 MVT --- User racpatel@redhat.com's account has been closed --- Additional comment from John Skeoch on 2015-04-20 05:25:23 MVT --- User racpatel@redhat.com's account has been closed --- Additional comment from Red Hat Bugzilla Rules Engine on 2015-11-27 05:15:34 EST --- This bug is automatically being proposed for the current z-stream release of Red Hat Gluster Storage 3 by setting the release flag 'rhgs‑3.1.z' to '?'. If this bug should be proposed for a different release, please manually change the proposed release flag.
REVIEW: http://review.gluster.org/15369 (cluster/dht: User xattrs value is not correct after brick stop/start) posted (#1) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: http://review.gluster.org/15369 (cluster/dht: User xattrs value is not correct after brick stop/start) posted (#2) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: http://review.gluster.org/15369 (cluster/dht: User xattrs value is not correct after brick stop/start) posted (#3) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: http://review.gluster.org/15369 (cluster/dht: User xattrs value is not correct after brick stop/start) posted (#4) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: http://review.gluster.org/15456 (cluster/dht: User xattrs value is not correct after brick stop/start) posted (#1) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: http://review.gluster.org/15456 (cluster/dht: User xattrs value is not correct after brick stop/start) posted (#2) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: http://review.gluster.org/15468 (cluster/dht: User xattrs value is not correct after brick stop/start) posted (#1) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: http://review.gluster.org/15468 (cluster/dht: User xattrs value is not correct after brick stop/start) posted (#2) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: http://review.gluster.org/15468 (cluster/dht: User xattrs value is not correct after brick stop/start) posted (#3) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: http://review.gluster.org/15468 (cluster/dht: User xattrs value is not correct after brick stop/start) posted (#4) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: http://review.gluster.org/15468 (cluster/dht: User xattrs value is not correct after brick stop/start) posted (#5) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: http://review.gluster.org/15468 (cluster/dht: User xattrs value is not correct after brick stop/start) posted (#6) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: http://review.gluster.org/15468 (cluster/dht: User xattrs value is not correct after brick stop/start) posted (#7) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: http://review.gluster.org/15468 (cluster/dht: User xattrs value is not correct after brick stop/start) posted (#8) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: http://review.gluster.org/15468 (cluster/dht: User xattrs value is not correct after brick stop/start) posted (#9) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: http://review.gluster.org/15468 (WIP cluster/dht: User xattrs value is not correct after brick stop/start) posted (#10) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: http://review.gluster.org/15468 (WIP cluster/dht: User xattrs value is not correct after brick stop/start) posted (#11) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: http://review.gluster.org/15468 (WIP cluster/dht: User xattrs value is not correct after brick stop/start) posted (#12) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: http://review.gluster.org/15468 (WIP cluster/dht: User xattrs value is not correct after brick stop/start) posted (#13) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: http://review.gluster.org/15468 (WIP cluster/dht: User xattrs value is not correct after brick stop/start) posted (#14) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: http://review.gluster.org/15468 (WIP cluster/dht: User xattrs value is not correct after brick stop/start) posted (#15) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: http://review.gluster.org/15468 (WIP cluster/dht: User xattrs value is not correct after brick stop/start) posted (#16) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: http://review.gluster.org/15468 (WIP cluster/dht: User xattrs value is not correct after brick stop/start) posted (#17) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: http://review.gluster.org/15468 (WIP cluster/dht: User xattrs value is not correct after brick stop/start) posted (#18) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: http://review.gluster.org/15468 (WIP cluster/dht: User xattrs value is not correct after brick stop/start) posted (#19) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: http://review.gluster.org/15468 (cluster/dht: User xattrs value is not correct after brick stop/start) posted (#20) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: http://review.gluster.org/15468 (cluster/dht: User xattrs value is not correct after brick stop/start) posted (#21) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: http://review.gluster.org/15468 (cluster/dht: User xattrs value is not correct after brick stop/start) posted (#22) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht: User xattrs value is not correct after brick stop/start) posted (#23) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht: User xattrs value is not correct after brick stop/start) posted (#24) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht: User xattrs value is not correct after brick stop/start) posted (#25) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht: User xattrs value is not correct after brick stop/start) posted (#26) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht: User xattrs value is not correct after brick stop/start) posted (#27) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht: User xattrs value is not correct after brick stop/start) posted (#28) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht: User xattrs value is not correct after brick stop/start) posted (#29) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht: User xattrs value is not correct after brick stop/start) posted (#30) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht: User xattrs value is not correct after brick stop/start) posted (#31) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht: User xattrs value is not correct after brick stop/start) posted (#32) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht: User xattrs value is not correct after brick stop/start) posted (#33) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht(WIP): User xattrs value is not correct after brick stop/start) posted (#34) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht(WIP): User xattrs value is not correct after brick stop/start) posted (#35) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht(WIP): User xattrs value is not correct after brick stop/start) posted (#36) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht(WIP): User xattrs value is not correct after brick stop/start) posted (#37) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht(WIP): User xattrs value is not correct after brick stop/start) posted (#38) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht(WIP): User xattrs value is not correct after brick stop/start) posted (#39) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht(WIP): User xattrs value is not correct after brick stop/start) posted (#40) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht(WIP): User xattrs value is not correct after brick stop/start) posted (#41) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht: User xattrs value is not correct after brick stop/start) posted (#42) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht: User xattrs value is not correct after brick stop/start) posted (#43) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht: User xattrs value is not correct after brick stop/start) posted (#44) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht: User xattrs value is not correct after brick stop/start) posted (#45) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht: User xattrs value is not correct after brick stop/start) posted (#46) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht: User xattrs value is not correct after brick stop/start) posted (#47) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht: User xattrs value is not correct after brick stop/start) posted (#48) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht(WIP) : User xattrs value is not correct after brick stop/start) posted (#49) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht(WIP) : User xattrs value is not correct after brick stop/start) posted (#50) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht(WIP) : User xattrs value is not correct after brick stop/start) posted (#51) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht(WIP) : User xattrs value is not correct after brick stop/start) posted (#52) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht(WIP) : User xattrs value is not correct after brick stop/start) posted (#53) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht : User xattrs value is not correct after brick stop/start) posted (#54) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht : User xattrs value is not correct after brick stop/start) posted (#55) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht : User xattrs value is not correct after brick stop/start) posted (#56) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht : User xattrs value is not correct after brick stop/start) posted (#57) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht : User xattrs value is not correct after brick stop/start) posted (#58) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht(WIP) : User xattrs value is not correct after brick stop/start) posted (#59) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht(WIP) : User xattrs value is not correct after brick stop/start) posted (#60) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht(WIP) : User xattrs value is not correct after brick stop/start) posted (#61) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht(WIP) : User xattrs value is not correct after brick stop/start) posted (#62) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht(WIP) : User xattrs value is not correct after brick stop/start) posted (#63) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht(WIP) : User xattrs value is not correct after brick stop/start) posted (#64) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht : User xattrs value is not correct after brick stop/start) posted (#65) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht : User xattrs value is not correct after brick stop/start) posted (#66) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht : User xattrs value is not correct after brick stop/start) posted (#67) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht : User xattrs value is not correct after brick stop/start) posted (#68) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht : User xattrs value is not correct after brick stop/start) posted (#69) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht : User xattrs value is not correct after brick stop/start) posted (#70) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht : User xattrs are not healed after brick stop/start) posted (#71) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht : User xattrs are not healed after brick stop/start) posted (#72) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht : User xattrs are not healed after brick stop/start) posted (#73) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht : User xattrs are not healed after brick stop/start) posted (#74) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht : User xattrs are not healed after brick stop/start) posted (#75) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht : User xattrs are not healed after brick stop/start) posted (#76) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht : User xattrs are not healed after brick stop/start) posted (#77) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht : User xattrs are not healed after brick stop/start) posted (#78) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
REVIEW: https://review.gluster.org/15468 (cluster/dht : User xattrs are not healed after brick stop/start) posted (#79) for review on master by MOHIT AGRAWAL (moagrawa@redhat.com)
COMMIT: https://review.gluster.org/15468 committed in master by Raghavendra G (rgowdapp@redhat.com) ------ commit 9b4de61a136b8e5ba7bf0e48690cdb1292d0dee8 Author: Mohit Agrawal <moagrawa@redhat.com> Date: Fri May 12 21:12:47 2017 +0530 cluster/dht : User xattrs are not healed after brick stop/start Problem: In a distributed volume custom extended attribute value for a directory does not display correct value after stop/start or added newly brick. If any extended(acl) attribute value is set for a directory after stop/added the brick the attribute(user|acl|quota) value is not updated on brick after start the brick. Solution: First store hashed subvol or subvol(has internal xattr) on inode ctx and consider it as a MDS subvol.At the time of update custom xattr (user,quota,acl, selinux) on directory first check the mds from inode ctx, if mds is not present on inode ctx then throw EINVAL error to application otherwise set xattr on MDS subvol with internal xattr value of -1 and then try to update the attribute on other non MDS volumes also.If mds subvol is down in that case throw an error "Transport endpoint is not connected". In dht_dir_lookup_cbk| dht_revalidate_cbk|dht_discover_complete call dht_call_dir_xattr_heal to heal custom extended attribute. In case of gnfs server if hashed subvol has not found based on loc then wind a call on all subvol to update xattr. Fix: 1) Save MDS subvol on inode ctx 2) Check if mds subvol is present on inode ctx 3) If mds subvol is down then call unwind with error ENOTCONN and if it is up then set new xattr "GF_DHT_XATTR_MDS" to -1 and wind a call on other subvol. 4) If setxattr fop is successful on non-mds subvol then increment the value of internal xattr to +1 5) At the time of directory_lookup check the value of new xattr GF_DHT_XATTR_MDS 6) If value is not 0 in dht_lookup_dir_cbk(other cbk) functions then call heal function to heal user xattr 7) syncop_setxattr on hashed_subvol to reset the value of xattr to 0 if heal is successful on all subvol. Test : To reproduce the issue followed below steps 1) Create a distributed volume and create mount point 2) Create some directory from mount point mkdir tmp{1..5} 3) Kill any one brick from the volume 4) Set extended attribute from mount point on directory setfattr -n user.foo -v "abc" ./tmp{1..5} It will throw error " Transport End point is not connected " for those hashed subvol is down 5) Start volume with force option to start brick process 6) Execute getfattr command on mount point for directory 7) Check extended attribute on brick getfattr -n user.foo <volume-location>/tmp{1..5} It shows correct value for directories for those xattr fop were executed successfully. Note: The patch will resolve xattr healing problem only for fuse mount not for nfs mount. BUG: 1371806 Signed-off-by: Mohit Agrawal <moagrawa@redhat.com> Change-Id: I4eb137eace24a8cb796712b742f1d177a65343d5
This bug was used to introduce a test case that sporadically(?) fails. Please have a look https://build.gluster.org/job/centos6-regression/6753/console and improve the test. Also note that tests are recommended to be placed under ./tests/bugs/<component>/ consider moving the test into a subdirectory. Having the tests in subdirectories makes it much easier to run all tests for a selected component.
Niels, I tried to execute multiple times on centos vm same test case but i did not get failure so i have not changed the same but after took centos-machine from nigels i tried the same, i got failure so now i have resolved the from below patch https://review.gluster.org/#/c/18436/ From above patch we are getting crash in one nfs case, i have tried to run same on centos machine in a loop but i did not get any crash, i have send a mail also to you. Please check and respond on the same. Thanks & Regards Mohit Agrawal
For specific to move the location of test cases i will update the same in next patch.
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.13.0, please open a new bug report. glusterfs-3.13.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution. [1] http://lists.gluster.org/pipermail/announce/2017-December/000087.html [2] https://www.gluster.org/pipermail/gluster-users/