Bug 982629 - quota: hitting "D" state for dd command
quota: hitting "D" state for dd command
Status: CLOSED ERRATA
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: glusterd (Show other bugs)
2.1
x86_64 All
high Severity high
: ---
: ---
Assigned To: krishnan parthasarathi
Saurabh
: ZStream
Depends On:
Blocks: 957685 988642
  Show dependency treegraph
 
Reported: 2013-07-09 08:46 EDT by Saurabh
Modified: 2016-01-19 01:12 EST (History)
10 users (show)

See Also:
Fixed In Version: glusterfs-3.4.0.34rhs
Doc Type: Bug Fix
Doc Text:
Previously, when data was being written into a quota enabled volume, using dd(1), the dd utility may hang on the write(3) system call. This can be confirmed, by using ps(1), that the process corresponding to the dd utility writing onto the volume, is in 'D' state. Now, with this update, we don't see this issue.
Story Points: ---
Clone Of:
: 988642 (view as bug list)
Environment:
Last Closed: 2013-11-27 10:27:23 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Statedump of the client process (158.85 KB, text/plain)
2013-07-11 10:09 EDT, vpshastry
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2013:1769 normal SHIPPED_LIVE Red Hat Storage 2.1 enhancement and bug fix update #1 2013-11-27 15:17:39 EST

  None (edit)
Description Saurabh 2013-07-09 08:46:35 EDT
Description of problem:
this is about creating data in a sub-directory.
while the disk quota gets exceeded and file is getting written, the dd command goes for a sleep mode.
and if try to come out of the script using ctrl-c then dd command goes for "D" state.
sometimes I am able to come out of the script and sometimes not. 

Version-Release number of selected component (if applicable):
client,
[root@rhsauto030 dir]# rpm -qa | grep gluster
glusterfs-3.4.0.12rhs.beta3-1.el6rhs.x86_64
glusterfs-fuse-3.4.0.12rhs.beta3-1.el6rhs.x86_64

server,
[root@quota1 ~]# rpm -qa | grep glusterfs
glusterfs-3.4.0.12rhs.beta3-1.el6rhs.x86_64
glusterfs-fuse-3.4.0.12rhs.beta3-1.el6rhs.x86_64
glusterfs-server-3.4.0.12rhs.beta3-1.el6rhs.x86_64


How reproducible:
many times has happened that the dd has gone for "D" state.

Steps to Reproduce:
1. create a volume, start it
2. enable quota
3. set quota on the volume for "30GB"
4. mount the volume using glusterfs
5. create 10 directories namely dir[1-10]
6. create 1 subdirectoy in each of them.
7. set quota limit for all of them, say for directories it is 2GB and for subdirectories it is 100MB
8. create data using below mentined script,

#!/bin/bash
set -x

create-data()
{
while [ 1 ]
do
cmd=`dd if=/dev/urandom of=$(date +%s) bs=1024 count=1024 2>&1`
echo $cmd
if [ "$(echo $cmd | awk '/Disk quota exceeded/')" ] 
then
   echo "quota limit reached"
   break
fi
done
return 1
}

create-data



Actual results:
many times the script does not come out of execution as the dd command has gone for sleep mode, for quite a time.

+++ date +%s
++ dd if=/dev/urandom of=1373347402 bs=1024 count=1024
+ cmd='dd: writing `1373347402'\'': Bad file descriptor
dd: closing output file `1373347402'\'': Bad file descriptor'
+ echo dd: writing '`1373347402'\'':' Bad file descriptor dd: closing output file '`1373347402'\'':' Bad file descriptor
dd: writing `1373347402': Bad file descriptor dd: closing output file `1373347402': Bad file descriptor
++ echo dd: writing '`1373347402'\'':' Bad file descriptor dd: closing output file '`1373347402'\'':' Bad file descriptor
++ awk '/Disk quota exceeded/'
+ '[' '' ']'
+ '[' 1 ']'
+++ date +%s
++ dd if=/dev/urandom of=1373347402 bs=1024 count=1024

[root@rhsauto030 ~]# ps -auxww | grep dd
Warning: bad syntax, perhaps a bogus '-'? See /usr/share/doc/procps-3.2.8/FAQ
root         2  0.0  0.0      0     0 ?        S    Jun19   0:00 [kthreadd]
root      4928  0.0  0.0  20216     8 ?        S    Jun19   0:00 hald-addon-input: Listening on /dev/input/event2 /dev/input/event0
68        4943  0.0  0.0  17800     8 ?        S    Jun19   0:00 hald-addon-acpi: listening on acpid socket /var/run/acpid.socket
root     26685  0.0  0.0 105180   780 pts/0    S+   10:53   0:00 dd if=/dev/urandom of=1373347402 bs=1024 count=1024
root     26695  0.0  0.0 103248   808 pts/1    S+   10:54   0:00 grep dd


now if I try to use ctrl-c to kill the process, the dd command goes to "D" state.

Expected results:
I percieve if the quota limit is reached, dd command should recieve response like "Disk quota exceeded" rather keep waiting and going to sleep mode.

Additional info:
Comment 5 vpshastry 2013-07-11 10:09:07 EDT
Created attachment 772255 [details]
Statedump of the client process

State dump of the client process is attached. It stuck at wb_flush.

[global.callpool.stack.1.frame.1]                                               
ref_count=1                                                                     
translator=fuse                                                                 
complete=0                                                                      
                                                                                
[global.callpool.stack.1.frame.2]                                               
ref_count=0                                                                     
translator=d-write-behind                                                       
complete=0                                                                      
parent=d-read-ahead                                                             
wind_from=ra_flush                                                              
wind_to=FIRST_CHILD (this)->fops->flush                                         
unwind_to=ra_flush_cbk
Comment 8 errata-xmlrpc 2013-11-27 10:27:23 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-1769.html

Note You need to log in before you can comment on or make changes to this bug.