Bug 1001453

Summary: Quota violated if a file is truncated to higher size
Product: [Red Hat Storage] Red Hat Gluster Storage Reporter: shylesh <shmohan>
Component: quotaAssignee: Raghavendra G <rgowdapp>
Status: CLOSED NOTABUG QA Contact: Saurabh <saujain>
Severity: high Docs Contact:
Priority: medium    
Version: 2.1CC: asriram, gluster-bugs, kparthas, mhideo, mzywusko, nsathyan, psriniva, rhs-bugs, rwheeler, saujain, storage-doc, storage-qa-internal, vagarwal, vbellur, vmallika
Target Milestone: ---Keywords: ZStream
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Known Issue
Doc Text:
Truncating a file to a larger size and writing to it violates the quota hard limit. This is because the XFS pre-allocation logic applied on the truncated file does not extract the actual disk space it consumed.
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-01-13 09:32:08 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 957769, 1020127    

Description shylesh 2013-08-27 05:48:19 UTC
Description of problem:
Quota will be violated if a file is truncated to larger size

Version-Release number of selected component (if applicable):
[root@gqac022 mnt]# rpm -qa| grep gluster
glusterfs-libs-3.4.0.20rhsquota5-1.el6rhs.x86_64
glusterfs-api-3.4.0.20rhsquota5-1.el6rhs.x86_64
glusterfs-devel-3.4.0.20rhsquota5-1.el6rhs.x86_64
glusterfs-3.4.0.20rhsquota5-1.el6rhs.x86_64
glusterfs-geo-replication-3.4.0.20rhsquota5-1.el6rhs.x86_64
glusterfs-api-devel-3.4.0.20rhsquota5-1.el6rhs.x86_64
glusterfs-rdma-3.4.0.20rhsquota5-1.el6rhs.x86_64
glusterfs-debuginfo-3.4.0.20rhsquota5-1.el6rhs.x86_64
glusterfs-fuse-3.4.0.20rhsquota5-1.el6rhs.x86_64
glusterfs-server-3.4.0.20rhsquota5-1.el6rhs.x86_64


How reproducible:
always

Steps to Reproduce:
1. Crated a 4 brick distribute volume
2. set the quota limit to 2GB
3. created 20 files of 100MB each on the mount point
4. now any further write operation fails saying "disk quota exceeded" which is   expected
5. increased the quota limit to 3GB
6. truncate any file to higher size 
truncate --size 10GB file1
7. now append any data to the file
for i in {1..100}
do
 cat /var/log/messages >>file1
done
8. check du -sh file1

Actual results:

Quota limit exceeded

Expected results:


Additional info:
[root@gqac022 mnt]# gluster v quota qtest list
                  Path                   Hard-limit Soft-limit   Used  Available
---------------------------------------------------------------
/                                 4.0GB 8977070246986897596      11.3GB  0Bytes


[root@gqac022 mnt]# du -sh 1
9.5G    1

[root@gqac022 mnt]# gluster v info
 
Volume Name: qtest
Type: Distribute
Volume ID: d13096d5-2ed4-4ed1-a3b1-f4e9d46761e7
Status: Started
Number of Bricks: 4
Transport-type: tcp
Bricks:
Brick1: gqac022.sbu.lab.eng.bos.redhat.com:/home/q1
Brick2: gqac023.sbu.lab.eng.bos.redhat.com:/home/q2
Brick3: gqac024.sbu.lab.eng.bos.redhat.com:/home/q3
Brick4: gqac025.sbu.lab.eng.bos.redhat.com:/home/q4
Options Reconfigured:
features.quota: on


Cluster info
===========
RHS nodes
------
gqac022.sbu.lab.eng.bos.redhat.com
gqac023.sbu.lab.eng.bos.redhat.com
gqac024.sbu.lab.eng.bos.redhat.com
gqac025.sbu.lab.eng.bos.redhat.com

Mount point
-------
gqac022.sbu.lab.eng.bos.redhat.com:/mnt


attached the sosreports

Comment 3 shylesh 2013-10-25 10:11:41 UTC
Truncating a file to a larger size and writing to it is a common used for image files used as a VM or additional disk, we should not be violating quota during such scenarios.

Comment 4 Saurabh 2013-10-25 11:09:25 UTC
Also, one can create a sparse file of size more than the quota limit set on the directory. 
Problem happens, that in case of continuous append operation one can cross the quota limits. As the file size keeps goes beyond quota limit set.

eg.
1. set a quota limit of 1GB on a directory
2. create sparse file of size 2GB with this command,
   time dd if=/dev/zero of=f.1 bs=1 count=1 seek=2097152k
3. then try to append data to it, using the command,
   while [ 1 ]; do cmd=`echo "hello" >> f.1`; if [ "$(echo $cmd | awk '/Disk quota exceeded/')" ]; then    echo "quota limit reached"; fi


result is the file size goes beyond 1GB

and in the present case it is,
[root@rhsauto005 dir1-data]# ls -l /mnt/nfs-test/sparse-file-dir/
total 264
-rw-r--r--. 1 root root 2147749299 Oct 25 08:01 f.1

i.e. around 2GB

Comment 5 Vivek Agarwal 2013-11-14 11:26:09 UTC
Moving the known issues to Doc team, to be documented in release notes for U1

Comment 6 Vivek Agarwal 2013-11-14 11:28:23 UTC
Moving the known issues to Doc team, to be documented in release notes for U1

Comment 7 Vivek Agarwal 2013-11-14 11:28:58 UTC
Moving the known issues to Doc team, to be documented in release notes for U1

Comment 9 Pavithra 2013-11-25 06:51:28 UTC
I;ve documented this as a known issue in the Big Bend Update 1 Release Notes. Here is the link:

http://documentation-devel.engineering.redhat.com/docs/en-US/Red_Hat_Storage/2.1/html/2.1_Update_1_Release_Notes/chap-Documentation-2.1_Update_1_Release_Notes-Known_Issues.html

Comment 10 Vijaikumar Mallikarjuna 2015-01-13 09:32:08 UTC
'ls -l can report the size of the sparse file which is pre-allocated'
To get the actual disk size of the file, please use 'du -ksh filename'

Closing as NOTABUG

Comment 11 Vijaikumar Mallikarjuna 2015-01-13 12:19:52 UTC
In gluster version 'glusterfs 3.4rhs' du command gives the actual disk spaces used.

Here is the test output:

root@rh1:/mnt # gluster --version
glusterfs 3.4rhs built on Jan 13 2015 17:25:03
Repository revision: git://git.gluster.com/glusterfs.git
Copyright (c) 2006-2011 Gluster Inc. <http://www.gluster.com>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GlusterFS under the terms of the GNU General Public License.

Testcase-2:

root@rh1:/mnt # g v quota vol1 limit-usage / 10MB
volume quota : success

root@rh1:/mnt # g v quota vol1 hard-timeout 0
volume quota : success

root@rh1:/mnt # g v quota vol1 soft-timeout 0
volume quota : success

root@rh1:/mnt # dd if=/dev/zero of=1 bs=1024k count=10
10+0 records in
10+0 records out
10485760 bytes (10 MB) copied, 0.110432 s, 95.0 MB/s

root@rh1:/mnt # gluster v quota vol1 list
                  Path                   Hard-limit Soft-limit   Used  Available  Soft-limit exceeded? Hard-limit exceeded?
---------------------------------------------------------------------------------------------------------------------------
/                                         10.0MB       80%      10.0MB  0Bytes             Yes                  Yes


root@rh1:/mnt # truncate --size 10GB 1

root@rh1:/mnt # for i in {1..100}; do cat /var/log/messages >> 1 ; done
cat: write error: Disk quota exceeded
cat: write error: Disk quota exceeded
cat: write error: Disk quota exceeded


root@rh1:/mnt # ls -l 1
-rw-r--r--. 1 root root 10000000000 Jan 13 17:30 1

root@rh1:/mnt # du -ks 1
10244	1




Testcase-2:
root@rh1:/mnt # time dd if=/dev/zero of=1 bs=1 count=1 seek=2097152k
1+0 records in
1+0 records out
1 byte (1 B) copied, 0.00551734 s, 0.2 kB/s

real	0m0.096s
user	0m0.002s
sys	0m0.004s

root@rh1:/mnt # ls -l 1
-rw-r--r--. 1 root root 2147483649 Jan 13 17:33 1

root@rh1:/mnt # du -sh 1
8.0K	1