| Summary: | Quota: stripe volume not showing the quota size properly | ||
|---|---|---|---|
| Product: | [Community] GlusterFS | Reporter: | Saurabh <saurabh> |
| Component: | quota | Assignee: | Junaid <junaid> |
| Status: | CLOSED WORKSFORME | QA Contact: | |
| Severity: | medium | Docs Contact: | |
| Priority: | medium | ||
| Version: | mainline | CC: | amarts, gluster-bugs, sgowda, vagarwal |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | Type: | --- | |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
|
Description
Saurabh
2011-04-14 13:33:00 UTC
PATCH: http://patches.gluster.com/patch/6901 in master (features/marker-quota: Use the actual disk-space consumed instead of ia_size returned in iatt for calculation of sizes.) PATCH: http://patches.gluster.com/patch/6902 in master (features/quota: use disk usage instead of file-sizes.) PATCH: http://patches.gluster.com/patch/6903 in master (cluster/stripe: aggregate xattrs holding quota-size in lookup and getxattr calls.) PATCH: http://patches.gluster.com/patch/6901 in master (features/marker-quota: Use the actual disk-space consumed instead of ia_size returned in iatt for calculation of sizes.) PATCH: http://patches.gluster.com/patch/6902 in master (features/quota: use disk usage instead of file-sizes.) PATCH: http://patches.gluster.com/patch/6903 in master (cluster/stripe: aggregate xattrs holding quota-size in lookup and getxattr calls.) [root@centos-qa-client-3 ~]# cd quota/inst/sbin/;./gluster volume quota ds2 list path limit_set size ---------------------------------------------------------------------------------- / 2097152 2641920 [root@centos-qa-client-3 sbin]# cd quota/inst/sbin/;./gluster volume quota ds2 list -bash: cd: quota/inst/sbin/: No such file or directory path limit_set size ---------------------------------------------------------------------------------- / 2097152 3170304 [root@centos-qa-client-3 sbin]# ./gluster volume info ds2 Volume Name: ds2 Type: Distributed-Stripe Status: Started Number of Bricks: 2 x 2 = 4 Transport-type: tcp Bricks: Brick1: 10.1.12.134:/mnt/ds2 Brick2: 10.1.12.134:/mnt/dds2 Brick3: 10.1.12.135:/mnt/ds2 Brick4: 10.1.12.135:/mnt/dds2 Options Reconfigured: features.limit-usage: /:2MB features.quota: on [root@centos-qa-client-3 sbin]# #################################################3 [root@centos-qa-client-1 gluster-test]# ls -lR .: total 560 drwxr-xr-x 2 root root 8192 Apr 19 02:21 d -rw-r--r-- 1 root root 512000 Apr 19 01:47 f.1 -rw-r--r-- 1 root root 512000 Apr 19 03:11 f.3 ./d: total 1056 -rw-r--r-- 1 root root 512000 Apr 19 02:20 f.1 -rw-r--r-- 1 root root 512000 Apr 19 02:21 f.2 -rw-r--r-- 1 root root 512000 Apr 19 02:21 f.3 -rw-r--r-- 1 root root 512000 Apr 19 02:21 f.4 [root@centos-qa-client-1 gluster-test]# outgrows the set limit, also per file the size differs from the mount point. [root@centos-qa-client-3 sbin]# ls -l /mnt/ds2 total 8 drwxr-xr-x 2 root root 4096 Apr 19 02:21 d [root@centos-qa-client-3 sbin]# ls -l /mnt/ds2/d total 792 -rw-r--r-- 1 root root 393216 Apr 19 02:20 f.1 -rw-r--r-- 1 root root 393216 Apr 19 02:21 f.2 -rw-r--r-- 1 root root 393216 Apr 19 02:21 f.3 [root@centos-qa-client-3 sbin]# ls -l /mnt/dds2/d total 756 -rw-r--r-- 1 root root 512000 Apr 19 02:20 f.1 -rw-r--r-- 1 root root 512000 Apr 19 02:21 f.2 -rw-r--r-- 1 root root 512000 Apr 19 02:21 f.3 [root@centos-qa-client-3 sbin]# PATCH: http://patches.gluster.com/patch/7197 in master (cluster/stripe: Fix a typo in strncmp.) PATCH: http://patches.gluster.com/patch/7348 in release-3.2 (cluster/stripe: Fix a typo in strncmp.) [root@centos-qa-client-2 ~]# /usr/sbin/gluster volume info dstripe Volume Name: dstripe Type: Distributed-Stripe Status: Started Number of Bricks: 2 x 2 = 4 Transport-type: tcp Bricks: Brick1: 10.1.12.134:/mnt/dstripe Brick2: 10.1.12.135:/mnt/dstripe Brick3: 10.1.12.134:/mnt/d-dstripe Brick4: 10.1.12.135:/mnt/d-dstripe Options Reconfigured: features.limit-usage: /:1MB features.quota: on [root@centos-qa-client-2 ~]# /usr/sbin/gluster volume quota dstripe list path limit_set size ---------------------------------------------------------------------------------- / 1048576 1056768 [root@centos-qa-client-2 ~]# ###############from client###################### [root@centos-qa-3 gluster-test]# stat f.3 File: `f.3' Size: 512000 Blocks: 1032 IO Block: 4096 regular file Device: 18h/24d Inode: 7077922 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 1969-12-31 16:00:00.000000000 -0800 Modify: 2011-06-01 01:16:49.000000000 -0700 Change: 2011-06-01 01:16:52.000000000 -0700 [root@centos-qa-3 gluster-test]# bc bc 1.06 Copyright 1991-1994, 1997, 1998, 2000 Free Software Foundation, Inc. This is free software with ABSOLUTELY NO WARRANTY. For details type `warranty'. 1032*512 528384 [root@centos-qa-3 gluster-test]# stat 1 File: `1' Size: 512000 Blocks: 1032 IO Block: 4096 regular file Device: 18h/24d Inode: 7077918 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 1969-12-31 16:00:00.000000000 -0800 Modify: 2011-06-01 01:09:40.000000000 -0700 Change: 2011-06-01 01:16:52.000000000 -0700 [root@centos-qa-3 gluster-test]# stat 2 File: `2' Size: 512000 Blocks: 1032 IO Block: 4096 regular file Device: 18h/24d Inode: 7077921 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 1969-12-31 16:00:00.000000000 -0800 Modify: 2011-06-01 01:09:27.000000000 -0700 Change: 2011-06-01 01:16:52.000000000 -0700 [root@centos-qa-3 gluster-test]# bc bc 1.06 Copyright 1991-1994, 1997, 1998, 2000 Free Software Foundation, Inc. This is free software with ABSOLUTELY NO WARRANTY. For details type `warranty'. 528384*3 1585152 [root@centos-qa-3 gluster-test]# ls -l total 792 -rw-r--r-- 1 root root 512000 Jun 1 2011 1 -rw-r--r-- 1 root root 512000 Jun 1 2011 2 -rw-r--r-- 1 root root 512000 Jun 1 2011 f.3 [root@centos-qa-3 gluster-test]# #####################################################3 [root@centos-qa-3 gluster-test]# touch f.4 touch: cannot touch `f.4': No such file or directory [root@centos-qa-3 gluster-test]# ls -l total 792 -rw-r--r-- 1 root root 512000 Jun 1 2011 1 -rw-r--r-- 1 root root 512000 Jun 1 2011 2 -rw-r--r-- 1 root root 512000 Jun 1 2011 f.3 [root@centos-qa-3 gluster-test]# vim f.4 [root@centos-qa-3 gluster-test]# ls -l total 800 -rw-r--r-- 1 root root 512000 Jun 1 2011 1 -rw-r--r-- 1 root root 512000 Jun 1 2011 2 -rw-r--r-- 1 root root 512000 Jun 1 2011 f.3 -rw-r--r-- 1 root root 8 Jun 1 2011 f.4 [root@centos-qa-3 gluster-test]# dd if=/dev/zero of=f.4 bs=512000 count=1 dd: closing output file `f.4': Disk quota exceeded [root@centos-qa-3 gluster-test]# ls -li total 928 7077918 -rw-r--r-- 1 root root 512000 Jun 1 2011 1 7077921 -rw-r--r-- 1 root root 512000 Jun 1 2011 2 7077922 -rw-r--r-- 1 root root 512000 Jun 1 2011 f.3 7077925 -rw-r--r-- 1 root root 131072 Jun 1 2011 f.4 [root@centos-qa-3 gluster-test]# ############# from server end ################### [root@centos-qa-client-2 ~]# /usr/sbin/gluster volume quota dstripe list path limit_set size ---------------------------------------------------------------------------------- / 1048576 1728512 [root@centos-qa-client-2 ~]# gluster> volume info str-marker
Volume Name: str-marker
Type: Distributed-Stripe
Status: Started
Number of Bricks: 2 x 2 = 4
Transport-type: tcp
Bricks:
Brick1: junaid-laptop:/export/str1
Brick2: junaid-laptop:/export/str2
Brick3: junaid-laptop:/export/str3
Brick4: junaid-laptop:/export/str4
Options Reconfigured:
diagnostics.brick-log-level: NONE
features.quota: on
features.limit-usage: /:100MB
#################################################
gluster> volume quota str-marker list
path limit_set size
----------------------------------------------------------------------------------
/ 104857600 0
#################################################
root@junaid-laptop:/mount# du -h /tmp/file1
3.2M /tmp/file1
root@junaid-laptop:/mount# cp /tmp/file1 file
gluster> volume quota str-marker list
path limit_set size
----------------------------------------------------------------------------------
/ 104857600 3289088
#################################################
root@junaid-laptop:/mount# rm file
gluster> volume quota str-marker list
path limit_set size
----------------------------------------------------------------------------------
/ 104857600 0
#################################################
root@junaid-laptop:/mount# for i in {1..10}; do cp /tmp/file1 file.$i; done
gluster> volume quota str-marker list
path limit_set size
----------------------------------------------------------------------------------
/ 104857600 32890880
##################################################
root@junaid-laptop:/mount# rm -rf *
root@junaid-laptop:/mount# dd if=/dev/zero of=file bs=1KB count=100
gluster> volume quota str-marker list
path limit_set size
----------------------------------------------------------------------------------
/ 104857600 106496
root@junaid-laptop:/mount# ls -lR
gluster> volume quota str-marker list
path limit_set size
----------------------------------------------------------------------------------
/ 104857600 110592
####################################################
So, what I am trying to convey is that when we are writing small files, the last write is not being received by the second stripe server which I confirmed by attaching gdb to the second stripe server. So, on next lookup the quota updation updates the xattr size. But in case of large file it works fine (I think even there one of the blocs is left out, but since it doesn't affect the overall output much).
I also changed 512*buf.ia_blocks to buf.ia_size in the code, and it works fine for small and large files. The below output is got after making the above said change.
gluster> volume quota str-marker list
path limit_set size
----------------------------------------------------------------------------------
/ 104857600 0
root@junaid-laptop:/mount# for i in {1..10}; do dd if=/dev/zero of=file.$i bs=1KB count=100; done
gluster> volume quota str-marker list
path limit_set size
----------------------------------------------------------------------------------
/ 104857600 1000000
what is the case of handling the quota limit on sparse files? If we consider buf->ia_size, then we may miss out on the holes, which is mostly the case with VMs. Currently, Quota doesn't treat sparse files as special files. It handles it by calculating the size using buf->ia_blocks x 512. (In reply to comment #13) > Currently, Quota doesn't treat sparse files as special files. It handles it by > calculating the size using buf->ia_blocks x 512. Exactly that is the point. We are calculating based on 'ia_blocks' and if we display using 'ia_size' there will be surely mismatches. Try it for a example file created like this: bash# dd if=/dev/urandam of=/mount/point/of/stripe-volume/file seek=20000 bs=4k count=1 The value is shown correctly on the next access to the file. |