Hide Forgot
Hello, I had volume with 2MB limit set and a directory with in it having 5MB limit set. Now , over fuse mount when some files are created the size shown is this, gluster> volume quota dist2 list path limit_set size ---------------------------------------------------------------------------------- / 2097152 1546240 /d1 5242880 1546240 /d2 5242880 0 Again, after removing all the files and making new files provides me size as, gluster> volume quota dist2 list path limit_set size ---------------------------------------------------------------------------------- / 2097152 1515520 /d1 5242880 1515520 /d2 5242880 0 but again for validation all files were removed, and new files were created, gluster> volume quota dist2 list path limit_set size ---------------------------------------------------------------------------------- / 2097152 0 /d1 5242880 0 /d2 5242880 0 gluster> gluster> gluster> volume quota dist2 list path limit_set size ---------------------------------------------------------------------------------- / 2097152 1612800 /d1 5242880 1612800 /d2 5242880 0 Point here is that there is a mismatch between the amount of data collected from client side and server side, after all this exercise. after the last trial the client(over fuse mount) says the total size as [root@centos-qa-client-1 d1]# bc bc 1.06 Copyright 1991-1994, 1997, 1998, 2000 Free Software Foundation, Inc. This is free software with ABSOLUTELY NO WARRANTY. For details type `warranty'. 5120*205 1049600 ./1024 1025 ./1024 1 and the extended attributes cumulatively show the same on servers. but the size displayed by the "list" command is wrong. Also, as per my observation for the last trial, when all files were removed and newly created just occupied 1MB(approx), the "Disk quota exceeded" message starts appearing, whereas the volume space limit is 2MB and it is just double of the size allowed to be occupied by new files. So, this error brings a confusion about how much before the space-quota getting filled the Warning for "Disk Quota exceeded" kind of message will start appearing?
*** Bug 2671 has been marked as a duplicate of this bug. ***
[root@centos-qa-client-1 d1]# bc bc 1.06 Copyright 1991-1994, 1997, 1998, 2000 Free Software Foundation, Inc. This is free software with ABSOLUTELY NO WARRANTY. For details type `warranty'. 1023*1024 1047552 is the total size of the 1023 files created under a dir over nfs mount. The same size is updated using the xattr on the backend. But the size field is again not updated properly. [root@centos-qa-client-2 sbin]# getfattr -m . -d -e hex /mnt/dist2 getfattr: Removing leading '/' from absolute path names # file: mnt/dist2 trusted.gfid=0x00000000000000000000000000000001 trusted.glusterfs.dht=0x0000000100000000000000007ffffffe trusted.glusterfs.quota.dirty=0x3000 trusted.glusterfs.quota.size=0x0000000000081400 [root@centos-qa-client-2 sbin]# [root@centos-qa-client-3 glusterfs]# getfattr -m . -d -e hex /mnt/dist2 getfattr: Removing leading '/' from absolute path names # file: mnt/dist2 trusted.gfid=0x00000000000000000000000000000001 trusted.glusterfs.dht=0x00000001000000007fffffffffffffff trusted.glusterfs.quota.dirty=0x3000 trusted.glusterfs.quota.size=0x000000000007e800 [root@centos-qa-client-3 glusterfs]# ############################################################################### [root@centos-qa-client-2 sbin]# ./gluster volume quota dist2 list path limit_set size ---------------------------------------------------------------------------------- /d1 1048576 1576960 /d2 1048576 0 / 1048576 529408
Inode context is not set on a directory on which a limit is set for a dist-rep volume, 1955 ret = quota_send_dir_limit_to_cli (frame, this, loc->inode, (gdb) s quota_send_dir_limit_to_cli (frame=0x2adb2c2255d0, this=0x431cb20, inode=0x2aaaaff0e15c, name=0x2aaab4003e90 "trusted.limit.list") at quota.c:1894 1894 int32_t ret = 0; (gdb) n 1895 char dir_limit [1024] = {0, }; (gdb) 1896 dict_t *dict = NULL; (gdb) 1897 quota_inode_ctx_t *ctx = NULL; (gdb) 1898 uint64_t value = 0; (gdb) 1900 ret = inode_ctx_get (inode, this, &value); (gdb) 1901 if (ret < 0) (gdb) 1924 return ret; (gdb) 1925 } [root@centos-qa-client-2 sbin]# ./gluster volume quota dist-rep2 list path limit_set size ---------------------------------------------------------------------------------- /d1 1.4MB [root@centos-qa-client-2 sbin]#
[root@centos-qa-client-2 sbin]# ./gluster volume quota dist2 list path limit_set size ---------------------------------------------------------------------------------- / 2097152 180000 /d1 1048576 80000 /d2 1048576 100000 /d2/dd2 102400 100000 /d1/dd1 102400 80000 [root@centos-qa-client-2 sbin]# although /d1/dd1 was also having same amount of data as /d2/dd2, seen for both fuse and nfs. glusterfs#10.1.12.134:/dist-rep2 on /mnt/gluster-dir type fuse (rw,allow_other,default_permissions,max_read=131072) 10.1.12.134:/dist2 on /mnt/nfs-test type nfs (rw,nfsvers=3,nolock,addr=10.1.12.134) glusterfs#10.1.12.134:/dist2 on /mnt/gluster-test type fuse (rw,allow_other,default_permissions,max_read=131072) [root@centos-qa-client-1 dd2]# pwd /mnt/gluster-test/d2/dd2 [root@centos-qa-client-1 dd2]# ls f.1 f.10 f.2 f.3 f.4 f.5 f.6 f.7 f.8 f.9 [root@centos-qa-client-1 dd2]# cd ../../d1/dd1 [root@centos-qa-client-1 dd1]# ls f.1 f.10 f.2 f.3 f.4 f.5 f.6 f.7 f.8 f.9
This bug is not consistently reproducible.
In this bug only the last case reported is inconsistently reproducible but all other cases happening were consistent,though after some patches and code correction those are not seen.
Well the problem I mentioned last has occurred again, [root@centos-qa-client-1 d2]# cd .. [root@centos-qa-client-1 gluster-test]# ls -R .: d1 d2 ./d1: f.1 f.13 f.18 f.22 f.27 f.31 f.36 f.40 f.45 f.5 f.54 f.59 f.63 f.68 f.72 f.77 f.81 f.86 f.90 f.95 f.10 f.14 f.19 f.23 f.28 f.32 f.37 f.41 f.46 f.50 f.55 f.6 f.64 f.69 f.73 f.78 f.82 f.87 f.91 f.96 f.100 f.15 f.2 f.24 f.29 f.33 f.38 f.42 f.47 f.51 f.56 f.60 f.65 f.7 f.74 f.79 f.83 f.88 f.92 f.97 f.11 f.16 f.20 f.25 f.3 f.34 f.39 f.43 f.48 f.52 f.57 f.61 f.66 f.70 f.75 f.8 f.84 f.89 f.93 f.98 f.12 f.17 f.21 f.26 f.30 f.35 f.4 f.44 f.49 f.53 f.58 f.62 f.67 f.71 f.76 f.80 f.85 f.9 f.94 f.99 ./d2: f.1 f.13 f.18 f.22 f.27 f.31 f.36 f.40 f.45 f.5 f.54 f.59 f.63 f.68 f.72 f.77 f.81 f.86 f.90 f.95 f.10 f.14 f.19 f.23 f.28 f.32 f.37 f.41 f.46 f.50 f.55 f.6 f.64 f.69 f.73 f.78 f.82 f.87 f.91 f.96 f.100 f.15 f.2 f.24 f.29 f.33 f.38 f.42 f.47 f.51 f.56 f.60 f.65 f.7 f.74 f.79 f.83 f.88 f.92 f.97 f.11 f.16 f.20 f.25 f.3 f.34 f.39 f.43 f.48 f.52 f.57 f.61 f.66 f.70 f.75 f.8 f.84 f.89 f.93 f.98 f.12 f.17 f.21 f.26 f.30 f.35 f.4 f.44 f.49 f.53 f.58 f.62 f.67 f.71 f.76 f.80 f.85 f.9 f.94 f.99 [root@centos-qa-client-1 gluster-test]# ls d1 | wc -l 100 [root@centos-qa-client-1 gluster-test]# ls -li d1/f.1 27722246 -rw-r--r-- 1 root root 1000 Apr 14 05:25 d1/f.1 [root@centos-qa-client-1 gluster-test]# ls -li d1/f.100 27724117 -rw-r--r-- 1 root root 1000 Apr 14 05:25 d1/f.100 [root@centos-qa-client-1 gluster-test]# ls d2 | wc -l 100 ############################################## [root@centos-qa-client-2 sbin]# ./gluster volume start dr2 Starting volume dr2 has been successful [root@centos-qa-client-2 sbin]# ./gluster volume quota dr2 list path limit_set size ---------------------------------------------------------------------------------- / 2097152 0 [root@centos-qa-client-2 sbin]# ./gluster volume quota dr2 list path limit_set size ---------------------------------------------------------------------------------- / 2097152 197000 [root@centos-qa-client-2 sbin]# ./gluster volume quota dr2 list path limit_set size ---------------------------------------------------------------------------------- / 2097152 198000 [root@centos-qa-client-2 sbin]# [root@centos-qa-client-2 sbin]# ./gluster volume quota dr2 list path limit_set size ---------------------------------------------------------------------------------- / 2097152 198000 [root@centos-qa-client-2 sbin]# ./gluster volume quota dr2 list path limit_set size ---------------------------------------------------------------------------------- / 2097152 198000 [root@centos-qa-client-2 sbin]# getfattr -m . -d -e hex /mnt/dr2 | grep size getfattr: Removing leading '/' from absolute path names trusted.glusterfs.quota.size=0x000000000000ee48 [root@centos-qa-client-2 sbin]# getfattr -m . -d -e hex /mnt/dr-dr2 | grep size getfattr: Removing leading '/' from absolute path names trusted.glusterfs.quota.size=0x000000000000ee48 [root@centos-qa-client-2 sbin]# getfattr -m . -d -e hex /mnt/dr-dr23 | grep size getfattr: /mnt/dr-dr23: No such file or directory [root@centos-qa-client-2 sbin]# ls /mnt/dr* /mnt/dr2: d1 d2 /mnt/dr23: d1 d2 /mnt/dr-dr2: d1 d2 [root@centos-qa-client-2 sbin]# getfattr -m . -d -e hex /mnt/dr23 | grep size getfattr: Removing leading '/' from absolute path names trusted.glusterfs.quota.size=0x0000000000014050 [root@centos-qa-client-2 sbin]# getfattr -m . -d -e hex /mnt/dr2/d | grep size d1/ d2/ [root@centos-qa-client-2 sbin]# getfattr -m . -d -e hex /mnt/dr2/d | grep size d1/ d2/ [root@centos-qa-client-2 sbin]# getfattr -m . -d -e hex /mnt/dr2/d1/ | grep size getfattr: Removing leading '/' from absolute path names trusted.glusterfs.quota.size=0x0000000000007530 [root@centos-qa-client-2 sbin]# getfattr -m . -d -e hex /mnt/dr23/d1/ | grep size getfattr: Removing leading '/' from absolute path names trusted.glusterfs.quota.size=0x000000000000a028 [root@centos-qa-client-2 sbin]# getfattr -m . -d -e hex /mnt/dr23/d2/ | grep size getfattr: Removing leading '/' from absolute path names trusted.glusterfs.quota.size=0x000000000000a028 [root@centos-qa-client-2 sbin]# getfattr -m . -d -e hex /mnt/dr2/d2/ | grep size getfattr: Removing leading '/' from absolute path names trusted.glusterfs.quota.size=0x0000000000007918 [root@centos-qa-client-2 sbin]# after creating data first time size has shown 197000, and very next attempt 198000 but after that it was intact. ########################################################################## But again when deleted the above mentioned data and recreated the data, that works fine, [root@centos-qa-client-2 sbin]# ./gluster volume quota dr2 list path limit_set size ---------------------------------------------------------------------------------- / 2097152 200000 [root@centos-qa-client-2 sbin]#
PATCH: http://patches.gluster.com/patch/6920 in master (features/marker-quota: fixes in self-healing of dirty inode code path.)
PATCH: http://patches.gluster.com/patch/6918 in master (marker/quota: use correct byte ordering when updating parent's size.)
bug 764403 is duplicated to this bug and this one is still happening, [root@centos-qa-client-1 gluster-test]# rm -rf * [root@centos-qa-client-1 gluster-test]# mkdir d1 d2 [root@centos-qa-client-1 gluster-test]# cd d1 [root@centos-qa-client-1 d1]# dd if=/dev/zero of=f.3 bs=1KB count=512 512+0 records in 512+0 records out 512000 bytes (512 kB) copied, 0.586422 seconds, 873 kB/s [root@centos-qa-client-1 d1]# cd ../d1 [root@centos-qa-client-1 d1]# cd ../d2 [root@centos-qa-client-1 d2]# dd if=/dev/zero of=f.3 bs=1KB count=512 512+0 records in 512+0 records out 512000 bytes (512 kB) copied, 0.539047 seconds, 950 kB/s [root@centos-qa-client-1 d2]# [root@centos-qa-client-2 sbin]# ls gluster glusterd glusterfs glusterfsd [root@centos-qa-client-2 sbin]# ./gluster volume quota drep disable Disabling quota has been successful [root@centos-qa-client-2 sbin]# pwd /root/quota/inst/sbin [root@centos-qa-client-2 sbin]# ls gluster glusterd glusterfs glusterfsd [root@centos-qa-client-2 sbin]# ./gluster volume quota drep enable Enabling quota has been successful [root@centos-qa-client-2 sbin]# ./gluster volume quota drep limit-usage / 2MB limit set on / [root@centos-qa-client-2 sbin]# ./gluster volume quota drep list path limit_set size ---------------------------------------------------------------------------------- / 2097152 0 [root@centos-qa-client-2 sbin]#
again for the above mentioned statement for bug764403, this time I tried to self heal using the command [root@centos-qa-client-1 gluster-test]# find . | xargs stat and the size was displayed accordingly, [root@centos-qa-client-2 sbin]# ./gluster volume quota drep list path limit_set size ---------------------------------------------------------------------------------- / 2097152 1040384 But this data is for dist-rep mount over fuse. Need to check for distributed volume and nfs mount.
PATCH: http://patches.gluster.com/patch/7010 in master (features/marker-quota: Add the disk space used to parent sizes during creation of new-files or symlinks.)
PATCH: http://patches.gluster.com/patch/7015 in master (features/marker-quota: handle creation of regular files using mknod.)
Hi Saurabh, Can we close this bug? regards, Raghavendra.
Hey Raghu, This one is also working fine.
Tested the issue for 2671 and found that on fuse mount the issue is not happening for distribute and distribute-replicate volumes. Tested the same over nfs mounts and its working properly.