Bug 1020886

Summary: quota: quota-deem-statfs on and their is mismatch in the values shown by "df -h" and quota list command
Product: [Red Hat Storage] Red Hat Gluster Storage Reporter: Saurabh <saujain>
Component: glusterdAssignee: vpshastry <vshastry>
Status: CLOSED ERRATA QA Contact: Saurabh <saujain>
Severity: high Docs Contact:
Priority: high    
Version: 2.1CC: grajaiya, kparthas, mzywusko, nsathyan, vagarwal, vbellur, vshastry
Target Milestone: ---Keywords: ZStream
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: glusterfs-3.4.0.37rhs-1 Doc Type: Bug Fix
Doc Text:
Previously, on a quota enabled volume, with quota-deem-statfs option being set to on, the disk usage statistics reported by df and volume quota list command, on a directory on which quota limit is configured, would differ. Now, with this update, the disk usage statistics reported by df and volume quota list are identical.
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-11-27 15:43:01 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Saurabh 2013-10-18 12:51:34 UTC
Description of problem:

On a volume the quota-deem-statfs option is switched to on.
This option is used in conjunction with quota. 
This helps the standard tools like "df" to display the size of the mount-point as per the quota limit set. 
The mount-point in consideration is that of a gluster volume.



Version-Release number of selected component (if applicable):
glusterfs-3.4.0.35rhs

How reproducible:
always

Steps to Reproduce:
1. create a volume, start it
2. enable quota, set limit of 1TB on root of the volume.
3. mount the volume over nfs.
4. start I/O on nfs mount-point
   here the for i/o , I created 2700 directories, set quota of 100MB in each and created files 1MB each.
5. stop the script creating i/o, effectivly stop i/o
6. set quota-deem-statfs on
7. gluster volume quota $volname list
8. df -h

Actual results:
[root@quota1 ~]# gluster volume quota dist-rep list /
                  Path                   Hard-limit Soft-limit   Used  Available
--------------------------------------------------------------------------------
/                                          1.0TB       80%     264.3GB 759.7GB
[root@quota1 ~]# 
[root@quota1 ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg_quota1-lv_root
                       44G  2.4G   40G   6% /
tmpfs                 4.9G     0  4.9G   0% /dev/shm
/dev/vda1             485M   32M  428M   7% /boot
/dev/mapper/RHS_vgvdb-RHS_lv1
                      421G  133G  289G  32% /rhs/brick1
localhost:dist-rep    1.0T  845G  180G  83% /tmp/dist-rep



[root@quota1 ~]# gluster volume info
 
Volume Name: dist-rep
Type: Distributed-Replicate
Volume ID: 1e06795e-7032-479d-9d48-026b832cede3
Status: Started
Number of Bricks: 6 x 2 = 12
Transport-type: tcp
Bricks:
Brick1: 10.70.42.186:/rhs/brick1/d1r1
Brick2: 10.70.43.181:/rhs/brick1/d1r2
Brick3: 10.70.43.18:/rhs/brick1/d2r1
Brick4: 10.70.43.22:/rhs/brick1/d2r2
Brick5: 10.70.42.186:/rhs/brick1/d3r1
Brick6: 10.70.43.181:/rhs/brick1/d3r2
Brick7: 10.70.43.18:/rhs/brick1/d4r1
Brick8: 10.70.43.22:/rhs/brick1/d4r2
Brick9: 10.70.42.186:/rhs/brick1/d5r1
Brick10: 10.70.43.181:/rhs/brick1/d5r2
Brick11: 10.70.43.18:/rhs/brick1/d6r1
Brick12: 10.70.43.22:/rhs/brick1/d6r2
Options Reconfigured:
nfs.addr-namelookup: on
features.quota-deem-statfs: on
features.quota: on



Expected results:
The values of the different fields should not have mismatch for "df" and quota list command.

Additional info:

Comment 2 Saurabh 2013-10-21 11:35:08 UTC
in one scenario I am just directories, no files in them
number of directories i 64000,

result is again, a mismatch by df and list command for "Used" field
[root@nfs1 ~]# df -h /tmp/dist-rep2
Filesystem            Size  Used Avail Use% Mounted on
localhost:dist-rep2   1.0T  615G  410G  61% /tmp/dist-rep2
[root@nfs1 ~]# 
[root@nfs1 ~]# 
[root@nfs1 ~]# 
[root@nfs1 ~]# gluster volume quota dist-rep2 list /
                  Path                   Hard-limit Soft-limit   Used  Available
--------------------------------------------------------------------------------
/                                          1.0TB       80%      0Bytes   1.0TB

Comment 3 Saurabh 2013-10-23 06:25:17 UTC
This is pretty nasty,

[root@quota1 ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg_quota1-lv_root
                       44G  2.3G   40G   6% /
tmpfs                 4.9G     0  4.9G   0% /dev/shm
/dev/vda1             485M   32M  428M   7% /boot
/dev/mapper/RHS_vgvdb-RHS_lv1
                      421G  141G  281G  34% /rhs/brick1
localhost:dist-rep3   1.0P  1.0P  174G 100% /tmp/dist-rep3
[root@quota1 ~]# 
[root@quota1 ~]# 
[root@quota1 ~]# gluster volume quota dist-rep3 list
                  Path                   Hard-limit Soft-limit   Used  Available
--------------------------------------------------------------------------------
/                                          1.0PB       80%      0Bytes   1.0PB


and I am not any data inside the volume

Comment 7 Saurabh 2013-11-04 13:37:06 UTC
tried to give a large value for quota limit-set,
[root@quota1 ~]# gluster volume quota dist-rep5 list
                  Path                   Hard-limit Soft-limit   Used  Available
--------------------------------------------------------------------------------
/dir                                       5.0GB       80%       5.0GB  0Bytes
/                                          4.0TB       80%      25.0GB   4.0TB


and df also displays the same

[root@quota1 ~]# df -h /var/run/gluster/dist-rep5
Filesystem            Size  Used Avail Use% Mounted on
localhost:dist-rep5   4.0T   26G  4.0T   1% /var/run/gluster/dist-rep5

also,
[root@quota1 ~]# gluster volume quota dist-rep5 list
                  Path                   Hard-limit Soft-limit   Used  Available
--------------------------------------------------------------------------------
/dir                                       5.0GB       80%       5.0GB  0Bytes
/                                          1.0TB       80%      25.0GB 999.0GB
[root@quota1 ~]# df -h /var/run/gluster/dist-rep5
Filesystem            Size  Used Avail Use% Mounted on
localhost:dist-rep5   1.0T   26G  999G   3% /var/run/gluster/dist-rep5


verified on,
glusterfs-3.4.0.34rhs-1

Comment 9 errata-xmlrpc 2013-11-27 15:43:01 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-1769.html