Bug 1146830

Summary: Enabling Quota on existing data won't create pgfid xattrs
Product: [Red Hat Storage] Red Hat Gluster Storage Reporter: Raghavendra G <rgowdapp>
Component: quotaAssignee: Vijaikumar Mallikarjuna <vmallika>
Status: CLOSED ERRATA QA Contact: Saurabh <saujain>
Severity: unspecified Docs Contact:
Priority: high    
Version: rhgs-3.0CC: amainkar, mzywusko, nbalacha, nsathyan, rgowdapp, rhs-bugs, sharne, smohan, storage-qa-internal, surs, vagarwal, vmallika
Target Milestone: ---Keywords: ZStream
Target Release: RHGS 3.0.3Flags: sharne: needinfo+
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: glusterfs-3.6.0.32-1 Doc Type: Bug Fix
Doc Text:
Previously, enabling Quota on Red Hat Storage 3.0 did not create pgfid extended attributes on existing data. The pgfid extended attributes are used to construct the ancestry path (from the file to the volume root) for nameless lookups on files. As NFS relies heavily on nameless lookups, quota enforcement through NFS would be inconsistent if quota were to be enabled on a volume with existing data. With this fix, the pgfid xattrs in the lookup on the existing data are healed.
 Story Points: ---
Clone Of:
: 1147378 (view as bug list) Environment:
Last Closed: 2015-01-15 13:40:22 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1087818, 1147378, 1147953, 1153907, 1162694    

Description Raghavendra G 2014-09-26 08:04:15 UTC 
Description of problem:
pgfid xattrs are used to construct the ancestry path in case of nameless lookups. rhs-2.1 had code to create pgfid xattr in lookup if it is not already there. However rhs-3.0 doesn't have pgfid healing code and hence pgfid xattrs won't be created on existing data. Since nfs relies on nameless lookups heavily, quota enforcement through nfs would be inconsistent.

[root@booradley 6]# gluster volume info
 
Volume Name: dist
Type: Distribute
Volume ID: 2b1406b1-cdb6-47b5-9d80-1cc035b73c5e
Status: Started
Number of Bricks: 1
Transport-type: tcp
Bricks:
Brick1: booradley:/home/export/dist1

[root@booradley dist1]# mount
localhost:dist on /mnt/glusterfs type fuse.glusterfs (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)

[root@booradley dist1]# ls /mnt/glusterfs/1/2/3/4/5/6/file 
/mnt/glusterfs/1/2/3/4/5/6/file

[root@booradley 6]# getfattr -d -m. -e hex /home/export/dist1/1/2/3/4/5/6/file 
getfattr: Removing leading '/' from absolute path names
# file: home/export/dist1/1/2/3/4/5/6/file
security.selinux=0x756e636f6e66696e65645f753a6f626a6563745f723a686f6d655f726f6f745f743a733000
trusted.gfid=0xa294cb915fd3412ab3ca2a3579e1820b

[root@booradley 6]# gluster volume quota dist enable
volume quota : success

[root@booradley 6]# getfattr -d -m. -e hex /home/export/dist/1/2/3/4/5/6/file 
getfattr: Removing leading '/' from absolute path names
# file: home/export/dist1/1/2/3/4/5/6/file
security.selinux=0x756e636f6e66696e65645f753a6f626a6563745f723a686f6d655f726f6f745f743a733000
trusted.gfid=0xa294cb915fd3412ab3ca2a3579e1820b
trusted.glusterfs.quota.11e18af2-ecda-4445-8fc7-e0f49d847b39.contri=0x0000000000000000

As can be seen above pgfid xattr is missing on file.

Version-Release number of selected component (if applicable):
rhs-3.0 and rhs-3.0.1 (rhs-2.1.x don't have this problem)

How reproducible:
always

Steps to Reproduce:
1. create some data on mount point
2. enable quota
3. check for pgfid xattr on backend

Actual results:
pgfid xattr not present.

Expected results:
xattr should be present.

Additional info:
Comment 1 Raghavendra G 2014-09-26 08:08:15 UTC 
However, on newly created files, xattr is created as can be seen below:

[root@booradley 6]# touch /mnt/glusterfs/1/2/3/4/5/6/newfile
[root@booradley 6]# ls /mnt/glusterfs/1/2/3/4/5/6/
file  newfile

[root@booradley 6]# getfattr -d -m. -e hex /home/export/dist1/1/2/3/4/5/6/newfile 
getfattr: Removing leading '/' from absolute path names
# file: home/export/dist1/1/2/3/4/5/6/newfile
security.selinux=0x756e636f6e66696e65645f753a6f626a6563745f723a686f6d655f726f6f745f743a733000

trusted.gfid=0x125f6c9c0a3f4c439baab1721119b1ba

trusted.glusterfs.quota.11e18af2-ecda-4445-8fc7-e0f49d847b39.contri=0x0000000000000000

trusted.pgfid.11e18af2-ecda-4445-8fc7-e0f49d847b39=0x00000001

As can be seen above trusted.pgfid.xxxx xattr is created on newfile.
Comment 3 Shalaka 2014-09-26 08:58:30 UTC 
Please review and sign-off edited doc text.
Comment 4 Vijaikumar Mallikarjuna 2014-10-29 10:26:39 UTC 
Patch submitted: https://code.engineering.redhat.com/gerrit/#/c/35659/
Comment 7 Saurabh 2014-11-21 11:39:29 UTC 
Executed on build glusterfs-3.6.0.33-1.el6rhs.x86_64
and found that pgfid is there for existing data.

[root@nfs1 ~]# getfattr -d -m . -e hex /rhs/brick1/d1r1/dir3/file_1156
getfattr: Removing leading '/' from absolute path names
# file: rhs/brick1/d1r1/dir3/file_1156
trusted.afr.vol0-client-0=0x000000000000000000000000
trusted.afr.vol0-client-1=0x000000000000000000000000
trusted.gfid=0x92cecd3ca3a24b1eb2ada1bd30af7a18

[root@nfs1 ~]# 
[root@nfs1 ~]# 
[root@nfs1 ~]# 
[root@nfs1 ~]# 
[root@nfs1 ~]# 
[root@nfs1 ~]# gluster volume quota vol0 enable
volume quota : success


[root@nfs1 ~]# ps -eaf | grep quota
root      1896  2735  0 06:14 pts/0    00:00:00 grep quota
root     24733     1  0 06:09 ?        00:00:00 /usr/sbin/glusterfs -s localhost --volfile-id vol0 -l /var/log/glusterfs/quota-mount-vol0.log -p /var/run/gluster/vol0.pid --client-pid -5 /var/run/gluster/vol0/
root     24740     1  0 06:09 ?        00:00:00 /usr/sbin/glusterfs -s localhost --volfile-id gluster/quotad -p /var/lib/glusterd/quotad/run/quotad.pid -l /var/log/glusterfs/quotad.log -S /var/run/22dccfe66cbd2039a66bb083faf48156.socket --xlator-option *replicate*.data-self-heal=off --xlator-option *replicate*.metadata-self-heal=off --xlator-option *replicate*.entry-self-heal=off
[root@nfs1 ~]# 
[root@nfs1 ~]# getfattr -d -m . -e hex /rhs/brick1/d1r1/dir3/file_1156
getfattr: Removing leading '/' from absolute path names
# file: rhs/brick1/d1r1/dir3/file_1156
trusted.afr.vol0-client-0=0x000000000000000000000000
trusted.afr.vol0-client-1=0x000000000000000000000000
trusted.gfid=0x92cecd3ca3a24b1eb2ada1bd30af7a18
trusted.glusterfs.quota.59edb169-9500-4b10-8e5d-4f857226f1a8.contri=0x0000000001b3d800
trusted.pgfid.59edb169-9500-4b10-8e5d-4f857226f1a8=0x00000001
Comment 9 errata-xmlrpc 2015-01-15 13:40:22 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-0038.html