Description of problem: Quota will be violated if a file is truncated to larger size Version-Release number of selected component (if applicable): [root@gqac022 mnt]# rpm -qa| grep gluster glusterfs-libs-3.4.0.20rhsquota5-1.el6rhs.x86_64 glusterfs-api-3.4.0.20rhsquota5-1.el6rhs.x86_64 glusterfs-devel-3.4.0.20rhsquota5-1.el6rhs.x86_64 glusterfs-3.4.0.20rhsquota5-1.el6rhs.x86_64 glusterfs-geo-replication-3.4.0.20rhsquota5-1.el6rhs.x86_64 glusterfs-api-devel-3.4.0.20rhsquota5-1.el6rhs.x86_64 glusterfs-rdma-3.4.0.20rhsquota5-1.el6rhs.x86_64 glusterfs-debuginfo-3.4.0.20rhsquota5-1.el6rhs.x86_64 glusterfs-fuse-3.4.0.20rhsquota5-1.el6rhs.x86_64 glusterfs-server-3.4.0.20rhsquota5-1.el6rhs.x86_64 How reproducible: always Steps to Reproduce: 1. Crated a 4 brick distribute volume 2. set the quota limit to 2GB 3. created 20 files of 100MB each on the mount point 4. now any further write operation fails saying "disk quota exceeded" which is expected 5. increased the quota limit to 3GB 6. truncate any file to higher size truncate --size 10GB file1 7. now append any data to the file for i in {1..100} do cat /var/log/messages >>file1 done 8. check du -sh file1 Actual results: Quota limit exceeded Expected results: Additional info: [root@gqac022 mnt]# gluster v quota qtest list Path Hard-limit Soft-limit Used Available --------------------------------------------------------------- / 4.0GB 8977070246986897596 11.3GB 0Bytes [root@gqac022 mnt]# du -sh 1 9.5G 1 [root@gqac022 mnt]# gluster v info Volume Name: qtest Type: Distribute Volume ID: d13096d5-2ed4-4ed1-a3b1-f4e9d46761e7 Status: Started Number of Bricks: 4 Transport-type: tcp Bricks: Brick1: gqac022.sbu.lab.eng.bos.redhat.com:/home/q1 Brick2: gqac023.sbu.lab.eng.bos.redhat.com:/home/q2 Brick3: gqac024.sbu.lab.eng.bos.redhat.com:/home/q3 Brick4: gqac025.sbu.lab.eng.bos.redhat.com:/home/q4 Options Reconfigured: features.quota: on Cluster info =========== RHS nodes ------ gqac022.sbu.lab.eng.bos.redhat.com gqac023.sbu.lab.eng.bos.redhat.com gqac024.sbu.lab.eng.bos.redhat.com gqac025.sbu.lab.eng.bos.redhat.com Mount point ------- gqac022.sbu.lab.eng.bos.redhat.com:/mnt attached the sosreports
Truncating a file to a larger size and writing to it is a common used for image files used as a VM or additional disk, we should not be violating quota during such scenarios.
Also, one can create a sparse file of size more than the quota limit set on the directory. Problem happens, that in case of continuous append operation one can cross the quota limits. As the file size keeps goes beyond quota limit set. eg. 1. set a quota limit of 1GB on a directory 2. create sparse file of size 2GB with this command, time dd if=/dev/zero of=f.1 bs=1 count=1 seek=2097152k 3. then try to append data to it, using the command, while [ 1 ]; do cmd=`echo "hello" >> f.1`; if [ "$(echo $cmd | awk '/Disk quota exceeded/')" ]; then echo "quota limit reached"; fi result is the file size goes beyond 1GB and in the present case it is, [root@rhsauto005 dir1-data]# ls -l /mnt/nfs-test/sparse-file-dir/ total 264 -rw-r--r--. 1 root root 2147749299 Oct 25 08:01 f.1 i.e. around 2GB
Moving the known issues to Doc team, to be documented in release notes for U1
I;ve documented this as a known issue in the Big Bend Update 1 Release Notes. Here is the link: http://documentation-devel.engineering.redhat.com/docs/en-US/Red_Hat_Storage/2.1/html/2.1_Update_1_Release_Notes/chap-Documentation-2.1_Update_1_Release_Notes-Known_Issues.html
'ls -l can report the size of the sparse file which is pre-allocated' To get the actual disk size of the file, please use 'du -ksh filename' Closing as NOTABUG
In gluster version 'glusterfs 3.4rhs' du command gives the actual disk spaces used. Here is the test output: root@rh1:/mnt # gluster --version glusterfs 3.4rhs built on Jan 13 2015 17:25:03 Repository revision: git://git.gluster.com/glusterfs.git Copyright (c) 2006-2011 Gluster Inc. <http://www.gluster.com> GlusterFS comes with ABSOLUTELY NO WARRANTY. You may redistribute copies of GlusterFS under the terms of the GNU General Public License. Testcase-2: root@rh1:/mnt # g v quota vol1 limit-usage / 10MB volume quota : success root@rh1:/mnt # g v quota vol1 hard-timeout 0 volume quota : success root@rh1:/mnt # g v quota vol1 soft-timeout 0 volume quota : success root@rh1:/mnt # dd if=/dev/zero of=1 bs=1024k count=10 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.110432 s, 95.0 MB/s root@rh1:/mnt # gluster v quota vol1 list Path Hard-limit Soft-limit Used Available Soft-limit exceeded? Hard-limit exceeded? --------------------------------------------------------------------------------------------------------------------------- / 10.0MB 80% 10.0MB 0Bytes Yes Yes root@rh1:/mnt # truncate --size 10GB 1 root@rh1:/mnt # for i in {1..100}; do cat /var/log/messages >> 1 ; done cat: write error: Disk quota exceeded cat: write error: Disk quota exceeded cat: write error: Disk quota exceeded root@rh1:/mnt # ls -l 1 -rw-r--r--. 1 root root 10000000000 Jan 13 17:30 1 root@rh1:/mnt # du -ks 1 10244 1 Testcase-2: root@rh1:/mnt # time dd if=/dev/zero of=1 bs=1 count=1 seek=2097152k 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00551734 s, 0.2 kB/s real 0m0.096s user 0m0.002s sys 0m0.004s root@rh1:/mnt # ls -l 1 -rw-r--r--. 1 root root 2147483649 Jan 13 17:33 1 root@rh1:/mnt # du -sh 1 8.0K 1