Bug 863100 - DHT :- inconsistent 'custom extended attributes',uid and gid, Access permission (for directories) if User set/modifies it after bringing one or more sub-volume down
DHT :- inconsistent 'custom extended attributes',uid and gid, Access permis...
Status: CLOSED DEFERRED
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: distribute (Show other bugs)
2.0
x86_64 Linux
medium Severity medium
: ---
: ---
Assigned To: Nithya Balachandran
Matt Zywusko
:
Depends On:
Blocks: 1064147
  Show dependency treegraph
 
Reported: 2012-10-04 08:43 EDT by Rachana Patel
Modified: 2015-11-27 05:06 EST (History)
8 users (show)

See Also:
Fixed In Version: glusterfs-3.4.0qa8, glusterfs-3.3.0.5rhs-42
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1064147 1286036 (view as bug list)
Environment:
Last Closed: 2015-11-27 05:06:05 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Rachana Patel 2012-10-04 08:43:26 EDT
Description of problem:
DHT :-  inconsistent 'custom  extended attributes',uid and gid, Access permission (for directories) if User set/modifies it after bringing one or more  sub-volume down

Version-Release number of selected component (if applicable):
3.3.0.3rhs-32.el6rhs.x86_64	

How reproducible:
always


Steps to Reproduce:
1. Create a Distributed volume having 3 or more sub-volumes on multiple server and start that volume.


2. Fuse Mount the volume from the client-1 using “mount -t glusterfs  server:/<volume> <client-1_mount_point>”

3. From mount point create some dirs and files inside it.

4. Bring on sub-volume down
[root@Rhs3 t1]# gluster volume status test
Status of volume: test
Gluster process						Port	Online	Pid
------------------------------------------------------------------------------
Brick 10.70.35.81:/home/t1				24009	Y	18564
Brick 10.70.35.85:/home/t1				24211	Y	16174
Brick 10.70.35.86:/home/t1				24212	Y	2360
NFS Server on localhost					38467	Y	2366
NFS Server on 10.70.35.81				38467	Y	12929
NFS Server on 10.70.35.85				38467	Y	10226
 
[root@Rhs3 t1]# kill -9 2360


5. 
Custom atribute:-
from mount point set custom attribute for directory and verify it on all server

client
[root@client test]# setfattr -n user.foo -v bar2 d1
[root@client test]# getfattr -n user.foo d1
# file: d1
user.foo="bar2"

server1:-
[root@Rhs1 t1]# getfattr -n user.foo d1
# file: d1
user.foo="bar2"

server2:-
[root@Rhs2 t1]# getfattr -n user.foo d1
# file: d1
user.foo="bar2"

server3:-
[root@Rhs3 t1]# getfattr -n user.foo d1
d1: user.foo: No such attribute

6.from mount point verify owner and group of dir and then modify it

[root@client test]# stat d1
  File: `d1'
  Size: 12        	Blocks: 2          IO Block: 131072 directory
Device: 15h/21d	Inode: 10442536925251715313  Links: 2
Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2012-10-04 12:10:57.006871636 +0530
Modify: 2012-10-04 12:10:57.006871636 +0530
Change: 2012-10-04 12:10:57.007864913 +0530

[root@client test]# chown u1 d1
[root@client test]# chgrp t1 d1
[root@client test]# stat d1
  File: `d1'
  Size: 12        	Blocks: 2          IO Block: 131072 directory
Device: 15h/21d	Inode: 10442536925251715313  Links: 2
Access: (0755/drwxr-xr-x)  Uid: (  500/      u1)   Gid: (  500/      t1)
Access: 2012-10-04 12:10:57.006871636 +0530
Modify: 2012-10-04 12:10:57.006871636 +0530
Change: 2012-10-04 12:13:05.168865621 +0530

7.verify that it is updated on all sub-volume except the down one

server1:-
[root@Rhs1 t1]# stat d1
  File: `d1'
  Size: 6         	Blocks: 8          IO Block: 4096   directory
Device: fc05h/64517d	Inode: 403116740   Links: 2
Access: (0755/drwxr-xr-x)  Uid: (  500/ UNKNOWN)   Gid: (  500/ UNKNOWN)
Access: 2012-10-04 12:10:57.006871636 +0530
Modify: 2012-10-04 12:10:57.006871636 +0530
Change: 2012-10-04 12:13:05.168865621 +0530

server2:-
[root@Rhs2 t1]# stat d1
  File: `d1'
  Size: 6         	Blocks: 8          IO Block: 4096   directory
Device: fc05h/64517d	Inode: 134423062   Links: 2
Access: (0755/drwxr-xr-x)  Uid: (  500/ UNKNOWN)   Gid: (  500/ UNKNOWN)
Access: 2012-10-04 12:10:56.807630951 +0530
Modify: 2012-10-04 12:10:56.807630951 +0530
Change: 2012-10-04 12:13:04.970323409 +0530

server3:-
[root@Rhs3 t1]# stat d1
  File: `d1'
  Size: 6         	Blocks: 8          IO Block: 4096   directory
Device: fc05h/64517d	Inode: 402655089   Links: 2
Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2012-10-04 12:10:51.424531338 +0530
Modify: 2012-10-04 12:10:51.424531338 +0530
Change: 2012-10-04 12:10:51.610007436 +0530

8.from mount point verify Dir permission and modify it

client
[root@client test]# stat d2
  File: `d2'
  Size: 12        	Blocks: 2          IO Block: 131072 directory
Device: 15h/21d	Inode: 9860248238918728119  Links: 2
Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2012-10-04 12:10:57.281276686 +0530
Modify: 2012-10-04 12:10:57.281276686 +0530
Change: 2012-10-04 12:10:57.282865329 +0530

[root@client test]# chmod 444 d2

[root@client test]# stat d2
  File: `d2'
  Size: 12        	Blocks: 2          IO Block: 131072 directory
Device: 15h/21d	Inode: 9860248238918728119  Links: 2
Access: (0444/dr--r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2012-10-04 12:10:57.281276686 +0530
Modify: 2012-10-04 12:10:57.281276686 +0530
Change: 2012-10-04 15:39:42.509942805 +0530

9. verify that it is updated on all sub-volume except the down one
server1
[root@Rhs1 t1]# stat d2
  File: `d2'
  Size: 6         	Blocks: 8          IO Block: 4096   directory
Device: fc05h/64517d	Inode: 134694359   Links: 2
Access: (0444/dr--r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2012-10-04 12:10:57.281276686 +0530
Modify: 2012-10-04 12:10:57.281276686 +0530
Change: 2012-10-04 15:39:42.509942805 +0530

server2
[root@Rhs1 t1]# stat d2
  File: `d2'
  Size: 6         	Blocks: 8          IO Block: 4096   directory
Device: fc05h/64517d	Inode: 134694359   Links: 2
Access: (0444/dr--r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2012-10-04 12:10:57.281276686 +0530
Modify: 2012-10-04 12:10:57.281276686 +0530
Change: 2012-10-04 15:39:42.509942805 +0530

server3:-
[root@Rhs3 t1]# stat d1
  File: `d1'
  Size: 6         	Blocks: 8          IO Block: 4096   directory
Device: fc05h/64517d	Inode: 402655089   Links: 2
Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2012-10-04 12:10:51.424531338 +0530
Modify: 2012-10-04 12:10:51.424531338 +0530
Change: 2012-10-04 12:10:51.610007436 +0530

10. now bring all sub-volumes up and perform lookup from client.

Verify updated 'custom  extended attributes',uid and gid, Access permission from client

11.verify access permission from sub-volume which was down previously
[root@Rhs3 t1]# stat d2
  File: `d2'
  Size: 6         	Blocks: 8          IO Block: 4096   directory
Device: fc05h/64517d	Inode: 134219638   Links: 2
Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2012-10-04 12:10:51.700461206 +0530
Modify: 2012-10-04 12:10:51.700461206 +0530
Change: 2012-10-04 12:10:51.759989493 +0530

 verify gid and uid on sub-volume which was down previously
[root@Rhs3 t1]# stat d1
  File: `d1'
  Size: 6         	Blocks: 8          IO Block: 4096   directory
Device: fc05h/64517d	Inode: 402655089   Links: 2
Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2012-10-04 12:10:51.424531338 +0530
Modify: 2012-10-04 12:10:51.424531338 +0530
Change: 2012-10-04 12:10:51.610007436 +0530

Verify custom  extended attributes for that directory.


server3:-
[root@Rhs3 t1]# getfattr -n user.foo d1
d1: user.foo: No such attribute


  
Actual results:
mount point shows modified values but on subvolumes values are not consistent

Expected results:
Once sub-volume is up it should update values for 'custom  extended attributes',uid and gid, Access permission (for directories).

Additional info:
Comment 2 shishir gowda 2012-10-19 00:21:57 EDT
There are multiple issues in this bug.
1. For user related xattrs, we can not handle healing as in dht, we would not be able to identify the correct copy. The work around for this is a subsequent setxattr for the same key, which will fix the xattr mis-match

2. UID/GID: A fix is in progress (862967)
3. mis-matching permission: Will investigate it respond back
Comment 4 Amar Tumballi 2013-02-15 07:11:59 EST
https://code.engineering.redhat.com/gerrit/#/c/1895/
Comment 5 Rachana Patel 2013-03-19 09:09:46 EDT
(In reply to comment #2)
> There are multiple issues in this bug.
> 1. For user related xattrs, we can not handle healing as in dht, we would
> not be able to identify the correct copy. The work around for this is a
> subsequent setxattr for the same key, which will fix the xattr mis-match
> 
> 2. UID/GID: A fix is in progress (862967)
> 3. mis-matching permission: Will investigate it respond back

1. If it is the case then, it should get documented.

2. It is dependent of bug 86296 and for that buf fixed in version is glusterfs-3.4.0qa5. So the same fix is available in latest build?

3. Could you please update on 3rd issue? what is the decision?
Comment 6 Scott Haines 2013-09-27 13:07:27 EDT
Targeting for 3.0.0 (Denali) release.
Comment 15 Susant Kumar Palai 2015-11-27 05:06:05 EST
Cloning this bug to 3.1. Will be fixed in future release.

Note You need to log in before you can comment on or make changes to this bug.